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20.  (Continued) 

(both  active  duty  and  retired)  were  used  as  subjects.  Their  comnents  and 
recommendations  as  to  the  operational  potential  of  the  options  matrix  were 
solicited  and  are  documented  herein. 

In  the  development  of  other  display  fornats,  two  decision  points,  i.e., 
selection  of  an  operations  area  and  selection  of  a transit  route,  were  in- 
vestigated. These  decisions  were  examined  to  determine  the  types  of  data 
required  to  arrive  at  a decision,  the  means  of  displaying  a variety  of  de- 
tailed information,  and  a way  to  present  the  data  in  a synoptic  form  which 
would  both  include  all  the  data,  and  provide  the  commander  with  a concise 
display  of  the  data  pertinent  to  his  decision. 
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EXECUTIVE  SUMMARY 


This  report  documents  the  continued  development  and  validation  of  Opera- 
tional Decision  Aids  (GDA)  conducted  for  the  office  of  Naval  Research.  The 
purpose  of  this  portion  of  the  study  included  the  performance  of  two  tasks: 

1.  Continue  the  work  previously  started  for  further  evaluation  of 
an  options  selection  matrix 

2.  Investigate  the  development  of  other  display  formats  as  aids  for 
decisions  which  are  not  adequately  supported  by  the  options  matrix. 

The  initial  effort  in  the  design  and  test  of  an  options  matrix  was  des- 
cribed in  the  previous  report  "Experimental  Evaluation  of  an  Options  Selec- 
tion Matrix  as  an  Operational  Decision  Aid, " and  discussed  the  technical  effort 
involved  in  the  design  and  the  initial  experimentation  with  the  options  matrix. 
The  report  documented  the  design  of  an  experiment  for  conducting  a pilot  study 
to  evaluate  the  use  of  increasing  levels  of  computer  support  and  provide  a 
commander  (subject)  with  an  information  presentation  which  would  aid  him  in 
making  a decision.  The  experiment  was  evaluated  using  personnel  of  the  Com- 
mand Support  Group  as  subjects. 

The  comments  and  recommendations  made  by  these  subjects  were  factored 
into  the  refinement  of  the  experiment  for  the  present  effort,  which  utilized 
senior  active  duty  officers  and  senior  retired  officers  presently  employed  at 
Grumman  to  validate  the  options  matrix  as  an  acceptable  decision  aid.  Twelve 
senior  officer  subjects  have  been  tested  with  the  revised  experimental  set-up. 

In  the  experiment,  a decision  point  was  selected  at  which  the  commander 
must  choose  a ship  to  detach  from  his  task  force  and  send  to  perform  a mission. 
This  tattletail  mission  involved  detaching  a ship  from  the  task  force  to  follow 
a potential  threat  ship  (e.g.,  a Kresta  with  long  range  SSN-3  surface  to  sur- 
face missile),  report  his  activities,  and  neutralize  the  threat  if  his  activity 
indicated  an  imminent  hostile  move  (i.e.,  radar  activity  indicative  of  a missile 
launch) . 

Scenarios  were  developed  around  the  ONRGDA  scenario  to  describe  the  situa- 
tion, and  the  subject  was  provided  with  sets  of  information  which  allowed  him 


to  select  one  of  five  candidate  ships  for  the  tattletail  mission.  The  experi- 
ment considers  three  conditions  of  computer  aiding: 

1.  Integrated  and  formatted  data  presentation,  which  represents  a first 
step  toward  automation,  wherein  data  are  computed,  formatted  and 
displayed  as  tabular  arrays  or  as  graphics.  There  is  no  computer 
assimilation  of  user  inputs. 

2.  Computer  integration  of  user  inputs,  in  which  a checklist  of  perti- 
nent factors  influencing  tattletail  selection  were  displayed  and  the 
test  subject  was  required  to  estimate  comparative  figures  of  merit 
for  the  candidate  ships. 

3.  Computer  solution  with  user  selected  variables  in  which  the  computer 
calculates,  integrates  and  displays  all  values  for  the  operator  to 
make  a final  decision  on  figures  of  merit  and  the  weighting  of  each 
tattletail  factor. 

For  the  first  part  of  the  experiment  (condition  l),  the  subject  was  pre- 
senter with  data  displays  which  provided  him  with  sets  of  information,  but 
did  not  provide  him  with  direction  as  to  what  data  was  most  pertinent  to  his 
tattletail  selection  decision. 

The  second  part  of  the  experiment  (condition  2)  provided  for  the  use  of 
the  graphic  aid,  namely  an  options  selection  matrix  and  a factors  table.  Hav- 
ing the  same  data  sets  (as  condition  l)  at  his  disposal,  he  was  guided  by  the 
factors  table  as  to  what  type  of  data  was  important  to  his  decision.  Based  on 
his  assessment  of  the  data  he  now  considered  pertinent,  the  subject  requested 
the  entry  of  figures  of  merit  into  the  matrix  for  each  of  the  candidate  ships, 
and  selected  a weight  for  each  of  the  factors  (i.e.,  acquisition  capability, 
surveillance  capability,  target  neutralization,  asset  survival  and  asset  im- 
portance to  own  force  defense)  which  must  be  considered  in  the  choice  of  a 
tattletail  ship.  Then,  the  computer  calculated  the  result  based  on  his  figures 
of  merit  and  weighting  factors. 

The  third  part  of  the  experiment  (condition  3),  provided  an  options  matrix 
for  which  a computer  solution  had  already  been  applied  and  the  figures  of  merit 


calculated  and  entered  into  the  matrix.  The  figures  of  merit  were  determined 
using  a set  of  algorithms  which  take  into  account  the  various  parameters 
affecting  the  selection  of  a tattletail  ship  (e.g.,  fuel  quantity/consumption, 
maximum  speed,  weapons,  CASREPTS,  intercepts,  etc.).  During  the  experiment, 
the  subject  had  the  option  of  accepting  the  computer  (school  book)  solution 
to  the  problem,  or  he  could  change  those  figures  of  merit  which  he  considered 
to  be  invalid.  Again  weights  were  subjectively  assigned  by  him  to  each  of  the 
tattletail  factors. 

To  provide  a realistic  setting  for  performing  the  tattletail  experiment, 
subject  testing  was  conducted  in  our  Command  Support  Simulator.  The  simula- 
tor includes  a commander's  console  from  which  the  subject  viewed  two  group 
view  displays  on  which  were  presented  the  graphic  and  tabular  data  necessary 
for  him  to  review  for  tattletail  selection.  He  also  had  available  two  small 
screen  displays  which  provided  him  with  the  capability  of  looking  at  up  to  four 
different  sets  of  information  at  the  same  time,  and  a track  ball  to  provide 
him  with  an  interactive  capability  with  the  graphic  display. 

Test  subjects  who  participated  in  the  experiment  are  representative  of 
the  people  who  would  be  expected  to  be  involved  in  operational  decision  mak- 
ing and  use  of  decision  aids  such  as  the  options  matrix.  In  performing  the 
experiment  we  were  most  interested  in  their  comments  as  to  the  operational 
potential  of  the  options  matrix,  and  accordingly  recorded  their  conversation 
both  during  and  after  the  experiment. 

In  the  condition  one  part  of  the  experiment,  subjects  tended  to  make 
quick,  but  relatively  inaccurate  decisions  for  the  tattletail  selection.  Sub- 
jects commented  that  there  was  too  much  data  to  consider,  and  therefore  deci- 
sions tended  to  be  made  based  on  looking  at  only  portions  of  the  data  available. 

The  use  of  the  options  matrix  and  a set  of  prompts  in  the  condition  two 
part  of  the  experiment  caused  the  subject  to  better  focus  on  the  data  pertinent 
to  the  tattletail  factor  he  was  considering.  Subjects  expressed  the  opinion 
that  determining  the  figures  of  merit  for  the  matrix  was  a somewhat  laborious 
process.  However,  they  agreed  that  selection  of  figures  of  merit  becomes  con- 
siderably easier  as  the  subject  gets  accustomed  to  using  them.  Most  subjects 
felt,  therefore,  that  with  practice  the  options  matrix  could  become  a useful 
tool  for  making  ship  selection  decisions. 
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Subjects  did  not  in  general  accept  the  use  of  an  options  matrix  filled 
in  by  a set  of  algorithms  as  used  in  the  condition  three  part  of  the  experi- 
ment. They  felt  that  a perferable  method  would  be  to  combine  conditions  two 
and  three,  so  that  parts  of  the  matrix  for  which  mathematical  answers  are 
readily  determined  are  filled  in  by  computer  generated  figures  of  merit,  and 
the  user  would  provide  the  figures  of  merit  for  factors  which  are  not  readily 
calculated. 

In  summary,  the  subjects  expressed  the  opinion  that  before  he  would  use 
the  options  matrix,  the  commander  would  have  to  acquire  trust  in  its  credibil- 
ity. First,  he  would  have  to  have  faith  in  the  system  analysts  who  generated 
the  parameters  and  the  equations  used  for  obtaining  the  figures  of  merit. 

Then  he  would  have  to  trust  the  operators  who  supply  the  input  data  used  by 
the  algorithms,  i.e.,  that  this  data  reflects  the  situation  and  is  properly 
inserted  into  the  equations.  Further,  he  would  have  to  trust  his  staff  that 
the  selections  made  by  the  use  of  the  matrix  are  indeed  the  best  selections 
to  make  and  reflect  the  best  choice  after  consideration  of  all  data  available. 

The  second  part  of  this  report  deals  with  the  means  to  provide  the  com- 
mander with  aids  for  decisions  which  are  not  adequately  supported  by  an  op- 
tions matrix.  Two  decision  points  which  had  been  previously  investigated  by 
a Grumman  generated  scenario  similar  to  the  ONRODA  scenario  were  selected. 
These  are  the  selection  of  an  operations  area  for  the  task  force  and  the  sel- 
ection of  a transit  route  to  the  area.  These  decisions  were  investigated  to 
detemine  the  types  of  data  required  to  arrive  at  the  decision,  the  means  for 
displaying  the  various  detailed  portions  of  the  information  and  a way  to  pre- 
sent the  data  in  a synoptic  form  which  would  both  include  all  the  data  and 
provide  the  commander  with  a concise  display  of  the  data  pertinent  to  his  de- 
cision. 

The  major  considerations  during  the  development  of  these  decision  aids 
were  to: 

• Allow  maximum  utilization  of  the  user's  judgement  and  skills  by 
implementing  an  interactive  system  which  permits  accepting, 
overriding,  or  altering  by  the  user  of  the  processed  data. 
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• Present  information  in  common  alphanumeric  and  graphic  formats  which 
expedite  user  interpretation. 

• Build  user  confidence  by  providing  the  user  the  option  to  call  up 
varying  levels  of  detailed  background  data  for  use  in  generating  a 
summary  presentation. 

• Permit  the  user  to  refer  directly  to  a computer  generated  solution 
for  specific  minor  decisions  in  stress  situations  when  speed  is 
crucial  and  work  load  intolerable. 

Although  the  decision  aids  developed  in  this  report  have  a broad  spectrum 
of  applications,  they  were  structured  around  solving  the  two  aforementioned 
decisions:  Best  Operations  Area  and  Best  Transit  Route,  centered  about  the 

ONRODA  scenario.  A step-by-step  description  of  their  operation  in  this  typi- 
cal setting  is  performed  to  demonstrate  user  interface. 

In  the  two  illustrative  examples  of  decision  aiding  techniques  described 
in  this  report,  the  user  is  provided  with  a series  of  visual  aids  that  he  may 
call  up  as  desired  to  assist  him  in  formulating  his  ultimate  decision.  These 
consist  of  interactive  alphanumeric  and  graphic  displays  in  which  data  germane 
to  the  problem  at  hand  has  been  retrieved  upon  command,  summarized  and  formatted 
for  quick,  precise  assimilation  by  the  user.  Included,  as  well,  is  the  capa- 
bility for  the  user  to  exercise  trial  solutions  and  analyze  their  outcome. 

A way  of  summarizing  numerical  data  associated  with  specific  geographical 
areas  is  the  dot  grid  in  which  the  density  of  dots  within  an  area  is  propor- 
tional to  the  numerical  value  to  be  represented.  This  type  of  data  display  ex- 
presses the  results  in  a clear,  concise  visual  form  that  can  quickly  be  inter- 
preted by  the  user,  and  can  be  used  by  a commander  when  a decision  must  be 
made  quickly. 

Of  additional  interest  to  the  tactical  commander  is  the  facility  to  solve 
and  display  the  results  of  relative  motion  vector  problems.  A computerized 
maneuvering  board  type  of  presentation  was  employed  which  provides  the  user 
with  immediate  comprehension  of  the  relative  motion  involved  in  hypothesized 
situations  and  gives  a visual  description  of  the  associated  time,  distance  and 
speed  parameters. 
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5. 


Although  these  data  displays  present  computer  processed  data  and  calcu- 
lated results  in  an  easy  to  comprehend,  visual  form,  they  do  not  formulate  any 
judgement  decisions.  The  user  controls  the  amount  of  computer  aiding  he  wishes, 
selects  the  input  data,  and  forms  his  own  decisions.  The  data  displays  can 
however  greatly  reduce  the  effort  that  would  normally  be  required  by  him  for 
data  retrieval,  organization,  condensation,  and  assimilation.  They  also  serve 
as  a checklist  to  ensure  that  he  explores  the  major  facets  pertaining  to  the 
decision-making  process. 

Although  the  display  formats  were  structured  around  so:ving  two  specific 
decision  problems,  they  have  a broad  spectrum  of  applications,  especially  where 
geographical  considerations  are  involved.  This  report  has  presented  data  dis- 
plays as  they  would  appear  and  be  used  by  operational  personnel.  The  next  step 
would  be  to  develop  new  software  and/or  modify  existing  software  necessary  to 
generate  these  displays  and  provide  the  interactive  capability.  At  this  point 
the  computer  sizing  requirements  and  the  cost  of  implementing  a demonstration 
model  could  be  determined.  The  concluding  step  would  be  to  implement  a demon- 
stration system  and  conduct  a validation  test. 


1 - EXPERIMENTAL  VALIDATION  OF  OPTIONS  SELECTION  MATRIX 


1.1  Introduction 

This  section  provides  a description  of  the  experimental  validation  of 
an  options  selection  matrix  using  the  ONRODA  scenario  and  a decision  point 
involving  the  selection  of  a tattletail  ship.  The  task  is  a follow-on  to 
work  previously  accomplished  and  documented  in  our  report  "Experimental  Eval- 
uation of  an  Options  Selection  Matrix  as  an  Operational  Decision  Aid."  Our 
previous  effort  focused  on  designing  the  experiment  and  perfoming  an  initial 
evaluation  using  personnel  of  the  Command  Support  Group  as  subjects.  For 
this  task  we  incorporated  the  comments  and  suggestions  of  these  subjects 
into  the  experiment,  and  have  now  conducted  the  experiment  using  senior  active 
duty  officers  and  senior  retired  officers  currently  employed  at  Grumman  as 
test  subjects. 

The  tattletail  scenario  chosen  for  the  experiment  is  within  the  context 
of  the  SRI  developed  ONRODA  scenario.  Test  scenarios  are  built  on  the  general 
scene  set  by  the  tattletail  scenario  and  are  used  for  conducting  the  experi- 
ment in  which  the  subject  is  asked  to  select  one  of  five  candidate  ships  for 
the  tattletail  mission.  Three  conditions  of  decision  aiding  are  considered 
in  the  experiment. 

The  experiment  itself  was  conducted  in  the  Command  Support  Simulator, 
which  houses  two  large  screen  and  two  small  screen  displays  for  the  subject's 
use  during  the  test  and  the  control  consoles  for  providing  the  appropriate 
displays.  The  experiment  was  designed  to  be  conducted  in  three  parts  to  per- 
mit evaluation  of  subject  performance  with  and  without  the  matrix  and  to  test 
for  acceptance  of  the  selection  matrix  as  a decision  aid.  The  three  conditions 
of  the  experiment,  the  graphic  and  tabular  data  which  are  available  to  the  sub- 
ject for  making  his  tattletail  decisions,  and  the  procedure  for  preparing  the 
subject  and  conducting  the  test,  are  described  in  detail. 

Test  data  collection  consisted  primarily  of  recording  the  conversation 
during  the  test.  Subsequent  playback  of  the  tapes  provided  data  on  elapsed 


times  between  events,  data  files  called  and  subject  comments.  Test  results 
are  documented  based  on  the  data  obtained  from  the  tape  recordings  and  from 
observations  made  and  noted  during  the  test. 

1.2  Subject  Selection 

The  experiment  was  performed  with  the  participation  of  twelve  subjects 
who  are  presently  on  active  duty  or  are  retired  senior  naval  officers.  The 
initial  plan  for  the  experiment  was  to  have  the  assistance  of  twelve  active 
duty  naval  officers  who  held  the  rank  of  Commander  and  above,  but  as  we  in- 
vestigated the  possibility  of  securing  the  services  of  active  duty  personnel, 
it  was  evident  that  it  would  be  difficult  to  obtain  the  required  number  and 
caliber  of  active  duty  personnel  to  participate  in  the  experiment.  We 
approached  the  resident  Naval  Plant  Representative  (NAVPR),  explained  our 
need  to  have  qualified  senior  active  duty  personnel  to  participate  in  the 
experiment  and  requested  the  services  of  any  of  the  NAVPR  who  met  our  re- 
quirements. We  gratefully  acknowledge  the  cooperation  and  services  of  the 
active  duty  personnel  who  so  willingly  participated  in  this  experiment. 

For  the  remainder  of  our  subjects  we  solicited  the  participation  of 
nine  retired  naval  officers  who  are  presently  employed  by  Grumman.  These 
officers  have  a combined  total  of  219  years  of  active  duty.  They  were  de- 
lighted to  have  the  opportunity  to  participate  in  the  experiment  and  to  share 
their  experience  to  help  in  the  decision  aiding  project. 

Of  the  twelve  subjects  who  participated  in  the  experiment,  seven  were 
captains  and  five  were  commanders.  They  have  a variety  of  naval  backgrounds. 
All  have  at  sea  experience  on  operational  staffs  in  a variety  of  capacities, 
including  CARGRU  staffs,  and  most  have  had  command  of  a ship,  including  car- 
riers with  flags  aboard. 

1.3  Tattletail  Scenario 

The  tattletail  scenario  chosen  for  the  experiment  is  described  within  the 
context  of  the  SRI  developed  ONRQDA  scenario,  which  is  synopsized  in  paragraph 
2.2  of  this  report.  It  assumes  that  there  are  two  main  forces  in  the  proximity 
of  ONRODA.  A Red  Force  (hostile^  located  approximately  200  miles  to  the  south- 
west of  the  island  is  composed  of  a Kresta  class  guided  missile  cruiser  (CLGM) 
and  five  destroyer  sir:ed  ships.  The  Kresta  has  a long  range  weapon  (the  SS-N-3 
cruise  missile)  which  can  be  brought  to  bear  against  the  Blue  (friendly)  forces. 
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For  the  purposes  of  the  experiment,  it  is  assumed  that  RA-5C  reconnaissance 
has  revealed  that  the  CLGM  has  detached  fran  the  remainder  of  the  Red  force 
and  is  proceeding  on  a course  of  010  degrees. 


J □ 
11 
D 
D 

n 

n 

n 

n 
' n 


! ' ••  . 


The  CTF  has  decided  that  the  CLGM  is  a potential  threat  to  the  ta6k  force 
and  has  directed  that  one  of  the  forces'  ships  be  detached  to  acquire  and  sur- 
veil  the  Red  ship.  Five  ships  have  been  identified  as  being  viable  candidates 
to  be  dispatched  to  follow  the  threat  ship.  It  is  the  task  of  the  subject  to 
select  the  best  ship  to  detach  to  conduct  the  tattletail  mission. 

The  Blue  Force  is  composed  of  two  Task  Groups  which  have  rendezvoused 
approximately  300  miles  west  of  ONRODA  to  form  Task  Force  1.  TF1  is  composed 
of  a total  of  14  ships  which  range  in  size  from  Carriers  (CV’s)  to  Ocean 
Escorts  (DE's)  with  varying  degrees  of  value  and  weapon  capability. 

1.4  Test  Scenarios 

The  tattletail  scenario  sets  the  general  scene,  introduces  the  subject 
to  the  forces  involved  and  establishes  the  requirement  for  a tattletail.  The 
test  scenarios  build  on  the  developed  scene  and  establish  a framework  for 
conducting  the  three  parts  of  the  experiment  described  in  Paragraph  1.6. 

To  permit  variation  of  the  scenarios  for  each  of  the  three  test  condi- 
tions comprising  the  experiment,  four  scenarios  were  generated  and  imple- 
mented during  the  experimental  investigations.  Each  scenario  uses  the  same 
basic  type  of  data,  i.e.,  graphics  of  the  problem  geometry  and  tabular  data 
representing  ships'  status  and  capabilities  for  the  tattletail  mission 
(See  Paragraph  1.7). 

There  are  five  candidate  tattletail  ships  in  each  scenario.  In  seme 
cases,  the  same  ship  may  be  in  more  than  one  scenario,  but  the  ship's  data 
has  been  changed. 

One  scenario  was  designed  to  be  used  solely  for  introducing  the  subject 
to  the  data  available  for  conducting  the  test.  This  scenario  is  presented  and 
explained  to  each  subject,  file  by  file,  for  the  purpose  of  acquainting  him 
with  the  information  he  can  use  in  the  course  of  the  test  to  make  his  tattle- 
tail selection.  No  specific  information  as  to  the  mission  constraints  is  pre- 
sented to  the  subject  with  this  scenario. 
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The  remaining  three  scenarios  are  designed  so  that  any  one  can  be  used 
for  any  of  the  three  test  conditions  which  provide  the  varying  levels  of  com- 
puter support.  Each  has  a set  of  specific  data  which  is  comparable  to  an 
operations  order  and  provides  the  subject  with  the  conditions  for  the  mission 
(e.g.,  allowable  time  to  establish  contact,  fuel  reserve  requirements,  area 
threats,  etc.).  There  is  a different  group  of  tattletail  candidate  ships  for 
each  scenario.  Where  ships  are  used  in  more  than  one  scenario,  the  ship's 
data  which  provides  equipment  fuel  and  ordnance  status  are  changed. 

Each  of  the  scenarios  is  designed  so  that  of  the  five  candidate  ships, 
it  is  relatively  simple  to  eliminate  two  of  them  as  not  being  able  to  meet 
one  of  the  critical  specific  conditions  required  of  the  mission  (e.g.,  the 
ship  cannot  intercept  the  Kresta  within  the  specified  time,  the  ship  does 
not  have  the  sufficient  fuel  to  accomplish  the  planned  mission,  etc.).  The 
remaining  ships  are  then  considered  for  their  relative  capabilities  to  per- 
form the  mission.  This  is  done  by  evaluating  the  basic  capability  of  each 
ship  and  the  degradation  in  its  capability  due  to  CASREFTS  (casualty  reports) 
to  the  various  equipments  which  are  necessary  for  mission  performance. 

1.5  Command  Support  Simulator  Operation 

As  a prelude  to  describing  the  specific  graphic  and  alphanumeric  data 
available  to  the  test  subject,  the  Command  Support  Simulator,  where  the  actual 
experiments  were  conducted,  is  herein  described. 

The  Command  Support  Simulator  is  a room  approximately  20'  x 20'  with  re- 
duced lighting  which  simulates  the  Commander's  area  of  a military  command 
center.  Figure  1-1.  From  the  test  subject's  point  of  view,  the  simulator 
consists  of  two  large  Group  View  Displays  (GVD's),  two  small  CRT  displays  which 
flank  two  operator  consoles,  and  a central  control  panel.  Data  is  stored  in  a 
Varian  V-73  computer  and  projected  on  the  displays  under  control  of  an  operator 
at  the  display  buffer  console  of  a Sanders  ADDS900  Display  Generator. 

For  the  purposes  of  the  Operational  Decision  Aids  experiments,  the  cap- 
abilities of  the  Command  Support  Simulator  were  utilized  to  provide: 
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• Graphic  display  of  the  tactical  picture  on  the  left  hand  GVD. 

The  interactive  capability  of  the  system  made  available  such 
features  as  graphic  pan  and  zoom,  and  hooking  of  candidate 
tattletail  tracks  to  provide  intercept  times  and  course  to  the 
ship  to  be  tailed.  In  addition,  this  screen  was  used  to  display 
the  options  matrix. 

• Tabular  data  files  display  on  the  right  hand  GVD.  These  dis- 
plays present  alphanumeric  information  of  ship's  status  and 
capability  as  well  as  tabulated  displays  of  calculated  data 
for  intercept  times  and  fuel  consumption. 

• Small  screen  display  of  the  data  tables  which  prompt  the  sub- 
ject to  look  at  certain  data  in  making  his  decision. 

The  operator  normally  displays  the  information  as  described  above,  but 
the  subject  can  call  for  the  data  to  be  displayed  on  any  combination  of 
screens  he  desires. 

1.6  Test  Conditions 

The  tattletail  experiment  was  designed  to  be  conducted  in  three  parts  to 
permit  evaluation  of  subject  performance  with  and  without  the  matrix,  and  to 
test  for  acceptance  of  the  selection  matrix  as  a decision  aid. 

In  condition  1 of  the  experiment,  data  are  presented  to  the  subject  in 
graphic  and  alphanumeric  format  on  two  group  view  displays  (GVD's),  and  on 
hard  copy  data  sheets.  The  graphic  data  is  similar  to  the  map,  track  and  inter- 
cept displays  available  in  the  Interim  Tactical  Flag  Command  Center  (ITPCC). 

It  provides  a map  background  of  the  ONRQDA  situation  and  the  relative  loca- 
tions of  underway  Blue  and  Red  forces.  This  display  also  provides  locational 
and  intercept  data  for  the  five  tattletail  candidate  ships  by  showing  the 
"present"  location  of  these  ships  and  the  track  each  would  take  in  order  to 
intercept  the  threat  ship  as  it  proceeds  along  its  expected  track. 

Alphanumeric  information  is  presented  as  a series  of  ITFCC  type  data  files 
which  provide  ships'  status  and  capabilities  data.  Tabular  displays  of  calcu- 
lated data  shewing  results  of  fuel  and  intercept  calculations  are  supplied  to 
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simulate  the  computational  support  available  in  the  ITFCC. 

Static  data  is  available  in  the  form  of  hard  copy  data  sheets.  This  is 
data  which  would  normally  not  be  expected  to  change  during  the  course  of  a 
deployment.  Included  are  ships  characteristics  summaries,  which  provide 
ship's  identification,  physical  configuration,  sensor/weapon  complement  and 
propulsion  equipment;  weapon  characteristics  which  provide  launch,  flight 
and  effectiveness  data;  and  tables  of  equipment  performance  data  which  pro- 
vide capabilities  of  sensor  and  weapon  systems. 

During  the  test,  the  subject  requests  any  of  the  graphic  or  tabular 
data  he  wishes  to  view.  The  operator  at  the  Sanders  display  console  calls 
up  the  data  and  it  is  displayed  for  the  subject.  The  subject  makes  requests 
based  on  his  judgement  as  to  the  most  pertinent  data  and  the  best  order  for 
looking  at  the  available  information. 

In  condition  2,  the  subject  is  provided  with  two  computer  aids  to  assist 
him  in  choosing  a tattletail  candidate.  The  first  is  a prompt  in  the  form  of 
a list  of  the  factors  and  sub-factors  which  are  important  to  consider  in  mak- 
ing the  tattletail  selection.  The  second  is  a selection  matrix  into  which  he 
enters  figures  of  merit  (FOM)  and  weighting  for  the  factors.  Essentially,  an 
FOM  is  the  subject's  estimate  (on  a scale  of  0 to  10)  of  a ship's  cap»bility 
to  perform  one  of  the  factors  (e.g.,  acquisition)  important  to  the  selection 
of  a tattletail,  and  weighting  is  his  estimate  (on  a scale  of  1 to  10)  of  the 
relative  importance  of  a factor  towards  meeting  the  tattletail  mission  crit- 
eria. 

The  factors/sub- factors  prompts  are  provided  to  aid  the  subject  in  estab- 
lishing an  orderly  data  search  pjattem.  Use  of  these  prompts  enables  him  to 
make  better  use  of  the  data  than  is  possible  under  test  condition  1.  The 
subject  calls  for  the  data  in  accordance  with  the  order  of  the  sub-factors 
list,  and  is  able  to  extract  all  the  information  required  to  fill  in  the 
selection  matrix  figures  of  merit  with  a minimum  of  interaction  with  the  data 
base. 

The  selection  matrix  is  the  means  by  which  the  figures  of  merit  obtained 
by  the  examination  of  the  available  data  are  recorded  for  use  in  making  the 
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tattletail  selections.  As  the  subject  selects  a value  for  each  of  the  figures 
of  merit,  he  verbally  requests  that  the  operator  enter  the  number  into  the 
matrix.  When  the  figure  of  merit  portion  of  the  matrix  is  complete,  the  sub- 
ject determines  the  weighting  for  each  of  the  factors.  He  then  makes  a tattle- 
tail  selection  based  on  the  weights  and  figures  of  merit  he  has  chosen.  A more 
complete  discussion  of  the  matrix,  including  sample  figures,  is  provided  in 
paragraph  1.8. 

After  the  subject  has  indicated  his  selection,  combinational  algorithms 
developed  as  part  of  the  school  book  solution  are  used  to  calculate  the  total 
figure  of  merit  for  each  ship,  based  on  the  weights  and  figures  of  merit  val- 
ues assigned  by  the  subject.  Results  are  then  compared  with  the  school  book 
solution.  The  school  book  solution  also  uses  the  subject's  weighting  factor 
inputs,  but  its  figure  of  merit  values  are  determined  by  algorithms  designed 
to  objectively  combine  factor  and  sub- factor  information  expected  to  be  con- 
tained in  a Command  and  Control  System  data  base. 

For  condition  3 of  the  experiment,  the  subject  is  shown  the  selection 
matrix  with  the  figures  of  merit  filled  in,  based  on  the  algorithms  developed 
for  the  school  book  solution.  He  is  asked  to  evaluate  the  figures  of  merit, 
to  again  review  the  available  data  as  he  has  done  in  the  other  two  conditions, 
and  to  change  any  figure  of  merit  with  which  he  disagrees.  The  subject  in- 
puts weighting  factors  as  in  condition  2.  The  combinational  algorithm  is 
then  used  to  determine  the  total  figure  of  merit  and  the  subject  is  asked  for 
his  feelings  about  the  school  book  solution.  The  object  is  to  determine 
whether  the  subject  is  "comfortable " with  a solution  over  which  he  merely 
has  "after  the  fact"  control,  whether  he  will  accept  the  school  book  solution, 
what  areas  of  the  matrix  he  is  most  likely  to  be  "distrustful"  of,  etc. 

1.7  Graphic  and  Tabular  Data 

In  conducting  the  experiment,  graphic  data  sufficient  to  provide  the  sub- 
ject with  the  geometry  of  the  situation  is  presented  on  the  left  hand  GVD. 
Figure  1-1  shows  a map  of  the  ONRCDA  area  on  the  left  GVD.  The  console  opera- 
tor can  pan  and  zoom  the  picture  to  shew  portions  of  the  situation  such  as  in 
Figure  1-2,  which  contains  en  enlargement  of  GNRGOA  Island  and  a portion  of  the 
Red  Force.  The  location  of  the  Blue  Task  Force  and  the  Red  Force  may  be  shown, 
then  the  display  zoomed  (Figure  1-3)  to  show  the  five  candidate  ships,  the 
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FIGURE  l_2  TYPICAL  PAN  AND  ZOOM  DISPLAY 


assumed  track  of  the  threat  ship  and  the  intercept  time  and  point  of  each 
ship  along  the  threat's  track.  As  this  graphic  data  is  presented,  a tabular 
display  on  the  right  screen  shows  data  pertinent  to  the  graphics. 

Representative  samples  of  tabular  data  are  shown  in  Figures  1-4  to  1-7. 
Formats  for  these  data  displays  were  chosen  to  be  similar  to  the  foimats 
which  are  used  in  the  ITFCC. 

The  Ships'  Positions  File  (Track  File)  of  Figure  1-4  shows  the  present 
positions  of  the  ships  which  are  in  the  Blue  Task  Force  and  in  the  opposing 
Red  Force.  For  each  ship  the  reference,  hull,  and  track  numbers  are  listed. 

The  latitude  and  longitude  of  each  ship,  present  course  and  speed,  and  source 
of  the  information  are  also  listed.  This  data  is  presented  in  conjunction  with 
the  graphic  display  presentation  which  shows  the  location  of  the  two  forces. 

The  Equipment  File  (Figure  1-5)  provides  information  on  the  equipment 
status  for  each  ship.  The  file  shows  the  ship's  name,  hull  number  and  refer- 
ence number.  The  equipments  shown  are  those  needed  to  perform  a tattletail 
mission  and  are  divided  into  electronics,  weapons,  and  propulsion.  For  each 
of  these  equipment  sets,  the  listing  contains  the  equipment  nomenclature  and 
type,  the  number  of  systems  of  each  kind,  the  status  of  each  system  (+  repre- 
sents a fully  operational  system,  0 represents  a fully  down  system,  and  a 
number  represents  the  percentage  capability  of  the  system),  the  date-time 
that  the  casualty  for  each  system  which  is  not  fully  operational  will  be  cor- 
rected, and  a brief  description  of  the  nature  of  the  casualty. 

The  CASREPTS  File  (Figure  1-6)  is  a summary  listing  of  the  ship's  cas- 
ualty reports.  The  equipment  file  shows  the  CASREPTS  for  each  one  of  the 
ships  listed  as  part  of  the  overall  ship's  equipments.  This  file  shows  the 
casualties  on  each  of  the  ships  as  a single  listing  and  provides  this  data  in 
a concise  form.  For  each  ship  casualty,  the  department,  nomenclature,  and 
typje  of  the  down  equipment  is  listed.  The  percent  operation  capability,  the 
casualty  correction  time  and  the  nature  of  the  casualty  of  each  down  equip- 
ment is  provided. 

Minimum  Time  Intercept  Calculation  (Figure  1-7)  provides  the  subject  with 
a calculation  of  the  time  it  will  take  each  of  the  candidate  ships,  proceeding 
at  maximum  sustained  spjeed,  to  intercept  the  threat  ship.  The  data  included 
in  this  display  provides  threat  ship  course  and  speed  data,  and  a tabulation 
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FIGURE  1-4  SHIPS  POSITIONS  FILE 
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FIGURE  1-5  EQUIPMENT  FILE 
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of  the  candidate  ships  showing  their  speed,  course  to  intercept  and  time  to 
intercept.  Fuel-consumption-to-intercept  data  is  provided,  along  with  a re- 
marks column  to  show  propulsion  CASREPTS  for  ships  with  long  intercept  times. 

Besides  the  data  samples  shown  in  Figures  1-4  to  1-7,  tabular  data  files 
have  been  developed  to  present  the  subject  with  data  on  candidate  ship's  loca- 
tion, ammunition  and  fuel  stores,  candidate  fuel  consumption  for  various  speeds 
used  in  the  tattletail  scenario,  and  candidate  fuel  consumption  to  perform  the 
specified  tattletail  mission. 

Four  sets  of  tabular  data  have  been  developed  for  the  experiment.  Each 
set  of  data  is  basically  the  same,  except  that  such  things  as  CASREPTS,  inter- 
cept calculation,  and  fuel  quantities/consumptions  are  changed  to  make  each 
scenario  different. 

1.8  Test  Procedure 

Prior  to  the  test,  each  subject  was  briefed  on  the  general  background  of 
the  ONR  Operational  Decision  Aid  Program  and  on  the  nature  of  the  tattletail 
experiment.  The  subject  was  given  a set  of  documents  to  read  in  preparation 
for  the  test.  These  included  a copy  of  the  tattletail  scenario  derived  from 
the  SRI  ONRODA  scenario,  a description  of  the  tattletail 's  task,  a general 
description  of  the  conditions  under  which  the  experiment  will  be  conducted, 
lists  of  the  types  of  data  which  will  be  available  to  the  subject,  a descrip- 
tion of  the  factors  involved  in  choosing  the  best  ship  to  send  on  the  assign- 
ment, and  a set  of  "hard  copy"  data.  The  latter  is  the  type  of  data  which 
normally  does  not  change  during  the  course  of  a mission  and  would  usually  be 
available  to  a commander  in  the  form  of  manuals,  tables,  characteristic  cards, 
etc. 

The  subject  was  requested  to  read  the  information  package  to  acquaint 
himself  with  the  overall  situation,  the  conduct  of  the  test  and  the  data  which 
is  available  for  making  a tattletail  selection. 

During  the  test,  the  subject  was  asked  to  occupy  the  CTF  chair  in  the  simu- 
lator (see  Figure  l-l)  from  which  he  can  view  the  two  group  view  displays  (GVD) 
and  the  small  screen  monitor  displays.  He  was  asked  if  he  had  questions  about 
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the  pre-test  data.  After  answering  any  questions,  the  Test  Conductor  presented 
the  sample  scenario  to  the  subject  by  having  the  console  operator  call  the 
graphic  and  tabular  data  for  display  on  the  GVD's,  and  by  explaining  to  the 
subject  the  information  displayed  by  each  graphic  display  or  tabular  file  of 
data.  This  presentation  acquainted  the  subject  with  the  type  of  data  which 
would  be  available  to  him  for  performing  the  tattletail  selection  and  pro- 
vided an  opportunity  to  ask  further  questions  if  he  did  not  understand  any 
part  of  the  presentation. 

Following  the  presentation  of  the  sample  scenario,  the  subject  was  given 
the  "Experiment  Specific  Conditions"  data  for  the  Condition  1 part  of  the  ex- 
periment. This  consists  of  two  typewritten  sheets  of  information  which  com- 
prise the  "operation  order"  for  the  experiment.  The  subject  was  asked  to 
first  read  the  specific  conditions.  He  then  could  request  a GVD  display  of 
the  graphic  or  tabular  formats  which  he  considered  pertinent  to  his 
tattletail  selection  for  Condition  1.  The  experiment  proceeded  with  essen- 
tially no  time  limits  on  the  subject. 

For  the  Condition  2 port  of  the  experiment,  the  subject  was  introduced 
to  the  Tattletail  Factors  (Figure  1-8),  which  showed  him  the  factors  and  sub- 
factors important  to  the  selection  of  a tattletail.  It  was  explained  that 
these  factors  should  be  considered  in  making  the  tattletail  selection  and  to 
aid  him.  The  files  and  elements  of  data  to  be  considered  for  each  factor  were 
displayed  as  a prompt  on  the  small  screen  display.  This  is  illustrated  in 
Figure  1-9  which  for  Surveillance  Effectiveness  shows  the  sub-factors  the  sub- 
ject should  consider  and  the  data  files  to  be  examined  for  this  data.  This 
figure  is  a photo  taken  at  the  operator's  console  and  shows  the  options  selected 
by  the  operator  with  a light  pen  in  conjunction  with  keyboard  function  keys, 
to  call  up  matrix  displays  (SH-(X),  total  and  A,  B,  C),  tabular  data  (A/N-l  and  2), 
and  graphics,  and  to  select  the  screens  on  which  the  display  is  to  appear  (tube). 

The  subject  was  then  shown  a blank  Tattletail  Selection  Matrix  (Figure 
1-10)  on  the  left  GVD,  and  it  was  explained  that  in  this  part  of  the  experi- 
ment he  would  be  asked  to  use  this  aid  in  making  the  tattletail  selection. 

The  method  of  filling  in  the  Figures  of  Merit  and  the  Weighting  Factors  was 
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FIGURE  1-8  TATTLETAIL  FACTORS 
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explained.  The  subject  filled  in  the  matrix  in  any  order  he  chose.  He  could 
change  his  values  at  any  time.  A new  set  of  "Experiment  Specific  Conditions" 
for  this  part  of  the  test  was  given  to  the  subject,  and  after  reading  it,  he 
proceeded  with  the  test.  He  requested  graphic/tabular  data  displays  and  used 
the  infomation  to  arrive  at  the  figure  of  merit  values  to  be  entered  in  the 
matrix.  Then,  based  on  his  knowledge  of  the  situation,  he  determined  the  weights 
for  each  of  the  factors.  The  test  conductor  then  had  the  subject  Inspect  the 
filled- in  matrix  and,  using  the  weights  and  figures  of  merit  he  had  entered, 
deteimine  the  ship  he  would  send  as  the  tattletail.  Figure  1-11  shows  two 
"filled  in"  matrices.  At  the  top  is  a typical  subject's  solution,  at  the  bot- 
tom is  the  theoretically  correct  solution  based  on  that  subject's  weighting  fac- 
tors. 

After  the  subject  made  his  choice  of  the  tattletail,  a combinational 
algorithm  was  used  to  determine  the  total  figure  of  merit  for  each  ship  using 
the  subject's  weighting  and  figures  of  merit.  His  choice  and  that  of  the 
combinational  algorithm  were  compared  and  discussed. 

For  the  condition  3 of  the  experiment,  the  subject  was  shown  a matrix 
already  containing  figures  of  merit  and  weights  for  each  factor.  It  was  ex- 
plained that  the  figures  of  merit  were  obtained  by  employing  a set  of  algor- 
ithms which  take  the  various  types  of  data  (which  were  available  to  the  sub- 
ject in  the  graphic  and  tabular  displays  and  in  hard  copy)  and  combine  them 
to  obtain  the  figure  of  merit,  and  that  the  weights  were  those  he  had  selected 
in  the  previous  part  of  the  test. 

The  combinational  algorithm  was  used  to  show  him  the  school  book  solution 
based  on  his  selection  of  weights  and  the  computer-generated  figures  of  merit. 

The  subject's  feelings  as  to  the  acceptability  of  the  school  book  solution  were 
requested  and  discussed.  Figure  1-12  shows  the  school  book  solution. 

At  this  time,  the  subject  was  asked  to  comment  on  the  three  conditions  of 
the  experiment  and  was  encouraged  to  use  his  knowledge  of  operational  condi- 
tions to  evaluate  the  use  of  the  matrix  as  an  operational  tool.  Qnphasis  was 
placed  on  determining  subject  acceptance  of  the  decision  aids. 
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FIGURE  1-12  MATRIX  WITH  SCHOOL  BOOK  SOLUTION 
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The  primary  means  for  collecting  data  during  the  experiment  was  by  means 
of  a voice  tape  recorder  situated  in  the  test  area  to  record  all  conversation 
during  the  course  of  the  test.  Subjects  were  encouraged  to  "think  out  loud" 
to  provide  a verbal  record  of  the  test,  and  tapes  were  generated  which  pro- 
vided information  of  test  elapsed  time,  data  files  called,  time  spent  on  each 
file,  and  subject's  comments. 

The  test  elapsed  time  was  recorded  to  provide  a record  of  the  times  when 
events  occurred  in  the  course  of  the  test.  Events  such  as  elimination  of  a 
ship  from  tattletail  consideration,  at  what  point  during  the  test  significant 
remarks  were  made,  and  the  time  taken  to  arrive  at  tattletail  ship  selection 
were  of  interest  in  evaluating  the  experiment. 

The  data  files  called  and  the  time  spent  on  each  file  provided  informa- 
tion on  the  subject's  methodology  in  arriving  at  a selection.  Recording  file 
call  up  permitted  the  examination  of  the  order  in  which  files  were  called, 
the  type  of  files  called  and  the  number  of  times  the  subject  looked  at  the 
same  file.  These  file  calls  were  then  evaluated  with  the  subject's  comments. 

Subject's  comments  during  the  test  provided  an  overall  insight  into  his 
performance.  Ccmments  gave  an  idea  of  the  subject's  understanding  of  the 
experiment,  his  grasp  of  the  data  presented  to  him,  his  feelings  for  the  ex- 
periment and  his  acceptance  of  the  decision  aid. 

1.10  Test  Results 

The  test  subjects  who  participated  in  this  experiment  are  representative 
of  the  people  who  would  be  expected  to  be  involved  in  operational  decision 
making  and  using  aids  such  as  the  options  matrix.  In  conducting  the  test  we 
recorded  data  as  explained  in  paragraph  1.9,  however  we  are  less  interested 
in  the  quantitative  data  obtained  than  we  are  in  the  qualitative  comments  the 
subjects  had  as  to  the  operational  potential  of  the  options  matrix. 

Prior  to  the  test,  each  subject  was  asked  to  ccmnent  on  the  descriptive 
material  he  had  been  given  to  prepare  him  for  the  test.  All  agreed  that  the 
material  gave  them  a good  understanding  of  what  the  test  was  about  and  of  the 
background  infomation  pertaining  to  the  scenario. 
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There  were  no  time  constraints  imposed  on  the  subject  to  accomplish  the 
tattletail  selection  for  the  three  conditions  of  the  experiment  or  to  discuss 
any  aspects  of  experiment  on  which  he  cared  to  cctnment.  Subjects  varied  in 
their  willingness  to  comment  and  to  share  their  experiences,  but  to  a large 
extent  they  tended  to  enjoy  the  opportunity  to  discuss  their  experiences  re- 
lated to  data  availability  to  the  commander,  display  of  the  data  and  the  abil- 
ity of  the  commander  to  make  intelligent  decisions  from  the  available  data. 
They  used  these  experiences  in  commenting  on  data  formats  in  the  experiment, 
the  use  of  the  options  matrix  and  the  mechanics  of  presenting  the  options 
matrix. 

The  sublets'  experience  in  some  cases  caused  them  to  be  predisposed  to 
ignore  the  ground  rules  imposed  as  part  of  the  experiment  and  to  use  their 
own  judgement  as  to  what  the  ground  rules  should  be.  For  example,  a ground 
rule  states  that  the  candidate  ship  must  be  capable  of  intercepting  the  track 
of  the  threat  ship  within  a period  of  12  hours,  and  the  Intercept  Calculation 
shows  the  time  to  intercept  for  each  of  the  five  candidate  ships.  This  data 
file  shows  that  because  of  initial  position  and  speed  capability  one  of  the 
ships  will  take  longer  than  the  required  12  hours  to  accomplish  the  inter- 
cept. Several  subjects  questioned  the  validity  of  the  imposition  of  a time 
constraint  of  this  nature  and  considered  that  it  would  not  be  the  type  of 
constraint  to  be  used  in  an  actual  situation.  It  was  explained  that  this 
time  constraint  was  used  to  help  bound  the  problem  in  order  to  allow  the  sub- 
ject the  ability  to  solve  the  problem  in  a simulated  environment  and  that  we 
fully  realized  that  sane  of  the  constraints  imposed  on  him  in  the  experiment 
were  artificial  but  were  done  to  facilitate  arrival  at  a solution. 

After  completing  the  condition  1 and  condition  2 parts  of  the  experiment, 
each  subject  was  asked  to  give  his  thoughts  about  each  part  and  then  to  com- 
pare the  two  parts. 

After  completing  the  condition  1 and  2 parts  of  the  experiment  each  sub- 
ject was  given  the  school  book  solution  for  the  best  tattletail  choice  and  why 
that  ship  should  be  chosen.  He  was  asked  to  comment  on  his,  versus  school 
book  choices  and  to  give  his  thoughts  about  each  of  the  two  parts  of  the  ex- 
periment. 
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For  these  two  conditions,  the  choices  of  tattletail  ships  were: 


Condition  1 


Condition  2 


Correct  choice  5 8 

Possible,  not  best  choice  6 3 

Incorrect  choice  1 1 

The  experiment  was  designed  so  that  for  each  scenario  there  are  two  or  three 
seemingly  best  tattletail  choices.  Use  of  the  matrix  helped  the  subjects  to 
make  the  correct  choice  more  often,  and  subjects  who  had  made  other  than  correct 
choices  tended  to  agree  with  the  school  book  solutions. 

Several  commented  that  the  tabular  data  presented  in  condition  1 contained 
too  much  information,  most  of  which  was  not  pertinent  to  solving  the  problem. 

In  addition,  the  data  pertaining  to  the  equipment  aboard  each  ship  would  have 
been  easier  to  read  if  it  were  presented  according  to  equipment  type  and  in 
such  a manner  that  the  same  equipment  typer  on  ships  could  be  compared  against 
each  other.  It  was  explained  that  the  data  as  it  was  presented  is  essentially 
the  way  it  apjpears  in  a system  presently  being  configured  for  a command  sup- 
port center.  Their  comments  were  certainly  valid  and  it  would  be  beneficial 
to  obtain  equipment  status  data  in  such  a manner  that  ships ' capabilities 
could  be  compared  against  each  other.  Another  recommendation  related  to  data 
presentation  was  to  show  only  the  data  piertinent  to  any  of  the  specific  tattle- 
tail factors  under  considration.  For  instance,  in  considering  Surveillance 
Effectiveness,  the  prompt  instructs  the  subject  to  look  at: 

• Candidate  ship's  maximum  speed  and  the  threat's  maximum  spieed 

• Fuel  consumption  calculation  for  percent  fuel  remaining  after  the 
mission 

• HF  radio  status  in  the  Ship's  Equipment  File 

• ESM  equipment  status  in  the  Ship's  Equipment  File. 

For  the  first  two  items  on  this  list,  the  data  for  each  of  the  ships  is 
shown  as  a single  display,  but  for  the  third  and  fourth  items  the  subject  must 
look  through  the  equipment  file  for  each  of  the  ships. 
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The  subjects  felt  that,  rather  than  show  the  complete  equipment  file, 
if  the  operator  could  query  the  file  to  provide  only  the  HF  or  ESM  equipment 
status  for  each  of  the  candidate  ships,  it  would  be  easier  to  compare  the 
ships  against  each  other  for  their  capabilities. 

The  subjects  realized  that  in  doing  the  condition  1 part  of  the  experi- 
ment they  could  look  at  the  available  data  in  any  order  they  desired,  but  that 
in  the  condition  2 part  of  the  experiment  they  were  prompted  to  look  at  cer- 
tain data  in  a predetennined  order,  (as  shown  above  for  the  Surveillance  Ef- 
fectiveness factor) . They  commented  that  the  use  of  the  matrix  with  the  prompts 
to  help  enter  figures  of  merit  into  the  matrix  was  a more  rigorous  approach  to 
obtaining  the  best  candidate  ship.  They  felt  that  the  matrix  was  cumbersome  to 
work  with  in  the  beginning,  but  as  they  progressed  in  filling  in  the  matrix 
they  were  able  to  obtain  a certain  amount  of  familiarity  with  the  procedure 
needed  to  make  figure  of  merit  selections  and  were  able  to  furnish  these  num- 
bers with  more  ease  and  confidence. 

The  average  time  to  complete  the  condition  1 part  of  the  experiment  was 
14  minutes,  25  seconds  and  for  condition  2 the  time  was  32  minutes,  30  seconds. 
Subjects  took  more  than  twice  as  long  to  complete  the  experiment  while  making 
use  of  the  matrix.  This  time  can  be  looked  at  in  conjunction  with  the  number 
of  files  the  subjects  called  which  were  an  average  of  8 files  for  condition  1 
and  12  files  for  condition  2.  The  subjects  looked  at  each  file  in  condition  1 
for  an  average  of  1 minute  48  seconds  and  in  condition  2 for  2 minutes  42  sec- 
onds. 

They  knew  that  it  took  a longer  time  to  fill  in  the  matrix  and  arrive  at 
a ship  selection  for  the  mission,  however  most  felt  that  their  selection  in 
condition  2 was  based  on  a more  thorough  evaluation  of  the  available  data  and 
were  more  confident  in  their  decision.  Subjects  expressed  the  opinion  that 
with  the  continued  use  of  an  aid  like  the  matrix  a facility  would  be  gained 
for  working  with  it,  so  that  the  time  required  to  reach  a decision  would  be 
comparable  to  that  without  the  use  of  the  aid.  More  importantly,  they  would 
be  more  confident  in  the  decision  being  correct  because  of  the  use  of  the 
matrix. 
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Two  subjects  expressed  the  opinion  that  they  would  prefer  not  to  work 
with  a "full "matrix  i.e.,  to  fill  in  figures  of  merit  for  all  the  ships  under 
consideration  and  then  to  use  the  matrix  figure  of  merit  selection  to  make 
the  choice  between  two  or  three  ships.  In  effect,  most  subjects  did  this  dur- 
ing the  course  of  the  experiment,  in  that  when  they  decided  that  one  or  more 
ships  would  not  meet  the  criteria  set  forth  in  the  ground  rules,  they  gave 
those  ships  figures  of  merit  of  zero  for  all  the  factors  and  disregarded  those 
ships  in  making  their  final  tattletail  selection.  One  subject  commented  "hav- 
ing a gut  feel  for  the  capabilities  of  the  ships  in  the  force  is  good,  but  it 
helps  to  have  this  backed  up  by  numbers  in  something  like  a matrix." 

The  above  comments  are  in  relation  to  the  subject's  filling  in  the  fig- 
ures of  merit  into  the  matrix  by  himself.  We  then  proceeded  to  condition  3 
of  the  experiment,  in  which  subjects  were  shown  a matrix  into  which  figures 
of  merit  had  been  inserted.  It  was  explained  that  the  figures  of  merit  appear- 
ing in  the  matrix  were  calculated  using  an  algorithm  for  each  factor  which  took 
into  consideration  a limited  set  of  the  same  computer  generated  and  hard  copy 
data  which  the  subject  had  available  to  him  during  the  condition  1 and  2 tests. 
Information  had  been  extracted  from  the  data  sets  and  inserted  into  the  al- 
gorithms to  yield  the  figures  of  merit  shown.  Subjects  were  asked  for  com- 
ments on  their  views  of  having  a computer  supplied  solution.  This  evoked  many 
comments  from  the  subjects. 

An  initial  reaction  from  most  subjects  was  to  question  if  the  figures  of 
merit  entered  in  the  cells  of  the  matrix  could  be  changed  or  if  they  were 
final.  It  was  explained  that  the  subject  was  free  to  look  at  the  data  again 
and  could  change  any  of  the  figures  of  merit  which  he  considered  to  be  incor- 
rect for  any  reason  and  that  total  figures  of  merit  would  be  recalculated. 

Subjects  showed  interest  in  how  the  algorithms  were  generated  and  what 
were  the  parameters  considered  in  order  to  arrive  at  the  algorithm.  Before 
accepting  the  algorithm  most  felt  that  they  would  want  to  be  able  to  under- 
stand the  algorithms  and  insure  themselves  that  the  algorithms  took  into 
account  a sufficient  set  of  data  to  adequately  cover  the  parameters  of  the 
problems.  They  felt  that  the  algorithms  could  take  into  account  more  informa- 
tion and  more  precisely  operate  on  it  than  they  could  and  would  represent  a 
great  advantage  if  the  algorithms  could  be  trusted. 
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This  led  to  discussions  of  how  the  algorithms  would  be  generated,  and  it 

was  generally  accepted  that  they  would  probably  be  initially  generated  by  a 

group  such  as  the  Command  Support  Systems  Group.  They  would  then  be  reviewed 
and  modified  by  the  members  of  the  CTF  staff  to  insure  that  they  included  all 
parameters  which  both  groups  considered  to  be  important. 

However,  this  did  not  lead  to  full  acceptance  of  the  matrix  figures  of 
merit  and  the  calculated  tattletail  ship  selection.  Most  subjects  felt  that 
they  would  question  the  individual  figures  of  merit  which  they  considered  to 
be  too  high  or  too  low  and  would  want  to  see  a justification  for  the  number. 

They  also  would  not  necessarily  accept  the  ship  with  the  highest  total  figure 

of  merit,  but  would  reserve  the  right  to  apply  their  own  "gut  feel"  to  the 

final  ship  selection.  The  feelings  of  most  subjects  were  to  a greater  or  lesser 
extent  summed  by  the  subject  who  stated  "I  wouldn't  allow  the  computer  to  make 
the  judgement  for  me". 

Subjects  differed  on  their  trust  of  numbers  in  the  matrix.  At  opposite 
ends  of  acceptance  were  two  subjects;  one,  an  engineer  stated,  "I  like  numbers 
and  have  faith  in  them",  the  other  stated,  "I  don't  think  what  you  put  up 
there  (in  the  matrix)  is  the  answer". 

Another  point  brought  up  with  regard  to  the  algorithms  concerned  the 
ability  to  introduce  into  the  equations  factors  to  account  for  specific  situa- 
tions as  they  arise  in  the  course  of  planning  for  the  mission.  This  would  in- 
clude things  such  as  abnormal  weather  conditions  and  new  types  of  enemy  weapons. 
Admittedly,  the  matrix  figures  of  merit  would  have  to  be  considered  in  light  of 
these  factors  and  the  individual  figures  of  merit  changed  to  account  for  these 
factors.  This  consideration  would  also  apply  to  the  weighting  of  factors  in- 
ternal to  the  algorithm  which  provide  weights  to  weapons  capabilities,  ships' 
worth,  type  warfare  expected,  etc. 

"The  matrix  takes  eveiything  into  account;  I couldn’t  do  that"  is  the  way 
one  subject  described  the  matrix.  Other  subjects  expressed  similar  opinions 
and  they  were  aware  that  even  when  using  the  prompts  of  condition  2 to  lead 
them  to  the  correct  data  to  use  for  the  figures  of  merit,  they  were  not  able 
to  integrate  the  data  mentally  to  arrive  at  a mathematical  figure  of  merit  as 
the  algorithms  do.  This  led  to  the  conclusion  reached  by  all  subjects  and  ex- 
pressed in  various  ways,  but  summed  up  by  the  comment  of  one  subject,  "The 
matrix  could  be  a tremendous  asset  for  evaluative  work." 
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The  subjects  agreed  that  the  evaluation  of  the  situation  and  the  manipu- 
lation of  data  in  the  matrix  is  the  type  of  effort  which  would  be  done  by 
staff  members  and  the  CTF  would  make  a decision  after  review  of  the  data. 

Most  subjects  felt  that  the  CTF  would  probably  not  be  satisfied  with  simply 
looking  at  a completed  matrix  and  allowing  the  matrix  results  provide  him 
with  the  solution.  He  would  probably  want  to  see  the  data  that  was  used  to 
obtain  the  figures  of  merit,  and  the  staff  would  have  to  be  prepared  to  justify 
these  numbers.  However,  one  of  the  subjects  thought  that  as  a commander  he 
would  be  inclined  to  accept  the  work  of  the  staff  members  in  obtaining  a sol- 
ution by  the  use  of  the  matrix,  saying,  "I  wouldn't  have  them  on  my  staff  if 
I didn't  trust  them."  Comments  of  the  subjects  indicate  that  the  use  of  the 
matrix  as  a decision  tool  would  depend  on  the  trust  the  commander  has  in  his 
staff  and  the  trust  the  staff  has  in  the  ability  of  algorithms  to  provide 
sufficiently  accurate  and  complete  data  for  filling  in  the  matrix. 

1.11  Conclusions 

Our  efforts  to  date  have  been  to  conduct  experiments  to  evaluate  the 
ability  of  a subject  to  use  an  options  selection  matrix  for  making  a decision. 
Twelve  subjects  with  operational  experience  have  been  tested  in  the  Command 
Support  Simulator  to  determine  their  ability  to  make  a decision  in  employing 
a specific  aid  and  to  elicit  their  comments  as  to  the  operational  potential 
of  the  options  matrix  decision  aid. 

The  experimental  testing  shows  that  using  the  matrix  and  set  of  prompts 
to  help  the  subject  look  for  data  pertinent  to  the  portion  which  he  is  asses- 
sing is  a rigorous  and  initially  somewhat  cumbersome  process.  However,  after 
using  the  matrix,  an  amount  of  familiarity  with  the  procedure  is  reached  and 
there  is  an  ease  in  using  the  aid  and  confidence  in  the  decision  reached. 

Subjects  expressed  the  opinion  that  with  practice  the  options  matrix  could 
become  a useful  tool  for  ship  selection  decisions. 

Subjects  did  not  accept  the  use  of  a matrix  filled  in  by  a set  of  algor- 
ithms, with  the  solution  of  the  best  ship  for  the  mission  displayed  as  an  "auto- 
matic" solution.  They  felt  that  in  some  way,  to  a greater  or  lesser  extent,  any- 
one working  with  the  matrix  would  want  to  understand  the  algorithms  used  in  the 
matrix  and  be  able  to  manipulate  the  inputs  into  the  algorithms  and  change  the 
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algorithms  to  obtain  confidence  in  the  outputs  provided  by  the  matrix.  A pos 
sible  approach  to  accomplish  this  would  be  implemented  by  presenting  the 
matrix  in  three  parts. 

• Factors  for  which  a definite  mathematical  answer  can  be  determined, 
such  as  acquisition  and  surveillance  capability  would  have  calcu- 
lated figures  of  merit  filled  in  by  algorithms. 

• Factors  for  which  a mathematical  solution  is  only  partially  accept- 
able, such  as  neutralization  and  survivability,  because  of  the 
number  of  variables  involved,  should  have  partial  solutions  derived 
and  presented.  These  can  be  examined  along  with  other  data  to 
arrive  at  revised  figures  of  merit. 

• Lastly,  for  factors  which  are  purely  subjective  and  have  no  mathe- 
matical solution,  e.g.,  commander  preferences,  the  matrix  will 
allow  for  entry  of  figures  of  merit  by  the  commander. 

In  summary,  the  subjects  expressed  the  opinion  that  to  use  the  matrix 
the  commander  would  require  trust.  This  would  be  initially  required  of  the 
system  analysts  who  determine  the  parameters  and  the  equations  used  for  ob- 
taining the  figures  of  merit.  Then  he  would  have  to  trust  the  operators 
who  supply  the  input  data  into  the  algorithms,  that  this  data  reflects  the 
situation  and  is  properly  inserted  into  the  equations.  He  would  have  to 
trust  his  staff  that  the  selections  made  by  the  use  of  the  matrix  are  in- 
deed the  best  selections  to  make  and  reflect  the  best  choice  after  consider- 
ation of  all  data  available. 


1-31 


II 


[J 

LI 


2 - INVESTIGATION  OF  OTHER  DISPLAY  FORMATS 

2.1  Introduction 

In  this  section  two  other  decision  problems  are  chosen  to  provide  a frame- 
work for  developing  other  data  display  formats  than  those  used  in  the  Tattle- 
tail  Selection  Problem.  The  two  decision  problems  are: 

• Selection  of  the  best  operating  area  for  the  Task  Force  to 
conduct  a specific  mission 

• Selection  of  the  best  transit  route  for  the  Task  Force  to 
reach  the  selected  operating  area. 

A sequence  of  interactive  graphic  and  alphanumeric  displays  are  described 
that  could  be  used  by  the  Task  Force  Commander  and  his  staff  to  assist  them  in 
arriving  at  a solution  to  the  two  aforementioned  decision  problems. 

First,  a synopsis  of  the  ONRODA  scenario  as  it  applies  to  the  two  deci- 
sion points  is  presented.  Next  the  Ops  Area  and  Transit  Route  Selection  prob- 
lems are  explained  in  more  detail,  including  a summary  of  the  major  data  dis- 
play formats  that  are  used  during  the  decision  making  process.  Following 
that  is  a description  of  how  a user  would  interface  with  the  computer  to 
enter  data,  call  up  data  displays,  respond  to  queries  and,  in  general,  util- 
ize the  capabilities  available  to  him.  Next,  examples  which  illustrate  in 
step-by-step  fashion  how  the  decision  aiding  capability  would  be  employed, 
first  to  solve  the  Ops  Area  Selection  problem  and  then  the  Transit  Route 
Selection  problem,  is  presented.  A proposed  experiment  to  test  the  effi- 
ciency and  user  acceptance  of  these  data  display  sequences  is  then  briefly 
discussed.  The  section  ends  with  concluding  remarks  and  recommendations  for 
future  work. 

2.2  ONRODA  Scenario  Synopsis 

The  analytic  effort  reported  herein  was  conducted  within  the  context 
of  the  ONRODA  Warfare  Scenario  described  in  SRI  Research  Memorandum  NWRC-RM-83. 
The  following  short  synopsis  of  the  ONRODA  scenario  is  presented  here  to  intro- 
duce the  situation  and  provide  a background  for  the  discussion  of  the  two  de- 
cision points  to  be  addressed. 
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Concurrent  with  the  emergence  of  nationalism  in  the  QNRGDA  Bay  littoral 
commencing  seme  thirty  years  ago,  the  peoples  of  the  area  have  been  exposed 
in  varying  degrees  to  the  influence  of  foreign  ideologies  and  politico-econ- 
omic pressures.  The  ultimate  result  has  been  that  some  factions  in  each 
emerging  country  (ultimately  Grey,  Yellow  and  Orange  nations)  have  fallen  under 
the  influence  of  Red,  and  others  under  that  of  Blue.  Although  the  trend  in 
each  has  been  toward  stronger  national  government,  the  ideological  alignments 
at  the  local  level  have  sometimes  been  inconsistent  with  the  posture  of  the 
national  government. 

At  the  level  of  national  government.  Grey  is  aligned  with  Blue,  Orange 
with  Red,  and  Yellow  is  weakly  aligned  with  Orange  and  Red. 

Grey  and  Orange  have  been  ideologically  opposed  and  hostile  to  each 
other  for  years. 

The  population  of  ONRODA  Island,  the  government  of  which  has  tradition- 
ally been  politically  aligned  with  Grey,  has  a significant  segment  of  Orange 
sympathizers.  This  minority  has,  within  the  past  year,  sent  delegations  to 
Orange  alleging  discriminatory  treatment  and  requesting  support  from  Orange 
for  their  cause. 

Commencing  with  a program  of  terrorism  instigated  by  a few  dissident 
local  leaders,  with  moral,  economic,  and  propaganda  support  from  Orange  and 
Red  sources,  a rebel  organization  known  as  "Greyhawks"  has  evolved  within 
Grey  nation  over  the  past  eight  years.  During  the  past  year  this  organiza- 
tion has  been  able  to  mount  an  armed  revolt  against  the  Grey  government,  with 
successes  including  the  capture  of  a major  city  in  Grey  two  months  ago,  and 
the  acquisition  of  military  control  over  the  areas  of  Grey  nation  shown  on 
the  area  chart  (See  Figure  2-3).  Hie  Greyhawks  have  sought  and  received  in- 
creasing support  from  Orange,  including  military  experts,  training,  small 
arms,  mortars,  and  rockets.  Materiel  has  been  delivered  from  Orange  by  air, 
water  and  overland  through  Yellow  (with  Yellow's  passive  cooperation). 

As  a result  of  Grey's  diminished  military  capability  and  preoccupation 
with  the  suppression  of  the  Greyhawk  rebellion,  Orange,  using  the  pretext  of 
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relief  for  the  "exploited"  ONRQDA  minority,  attacked  the  Grey  forces  on  ONRODA 
Island  on  31  December.  By  nightfall,  the  Island  was  held  by  Orange  forces. 

The  Orange  incursion  onto  ONRQDA  was  complete  with  the  introduction  of  SAMs, 
Anti-aircraft  guns,  and  construction  equipment  on  the  day  the  island  was  seized. 
Grey's  counter-action  to  the  ONRQDA  landing  by  Orange,  which  was  in  the  fom  of 
air  strike  from  Grey  mainland,  was  too  late  to  preclude  the  establishment  of  an 
effective  air  defense  of  the  island  by  Orange  forces.  Orange  imnediately  be- 
gan construction  efforts  to  enhance  the  military  capability  of  the  existing 
civilian  airport  on  ONRODA. 

Grey  filed  an  immediate  (31  December)  urgent  protest  to  the  UN  Security 
Council  requesting  intercession  in  her  behalf,  and  similarly  appealed  to  Blue 
for  unilateral  assistance.  Blue  political  analysts'  evaluation  was  that  the 
UN  would  not  take  any  timely  effective  action.  Furtheimore,  it  was  deemed 
unlikely  that  Bine  Congress  would  act  on  the  Grey  request  in  less  than  sev- 
eral weeks.  That  Blue  President  would  order  direct  intervention  in  the  ab- 
sence of  a Congressional  mandate  was  considered  doubtful  at  this  time. 

A Blue  carrier  Task  Force  has  been  formed  and  has  been  given  the  mission: 
"When  directed,  begin  operations  to  neutralize  Orange  forces  and  facilities 
on  ONRODA  Island  in  order  to  defend  Grey.  Do  not  attack  targets  on  Orange 
mainland  or  in  Orange  ports.  Take  defensive  measures  to  protect  your  force 
from  Orange  or  Red  retaliations." 

As  a result  of  increased  Orange  and  Grey  military  activity  in  November 
and  December,  Blue  increased  her  forces  in  the  area  so  that  on  the  morning  of 
31  December  one  carrier  group  (CARGRU-ONE)  was  conducting  training  exercises 
about  400  miles  west  of  Mid-Ocean  Island  and  another  carrier  group  (CARGRU- 
TWO)  was  in  various  liberty  ports  about  1500  miles  from  ONRODA  Island. 

On  the  evening  of  31  December  COMBLUFLT  directed  CCMCARGRU- ONE  to  posi- 
tion CARGRU-ONE  within  strike  distance  of  ONRQDA  Island  and  the  Grey  coast 
but  to  remain  well  out  of  the  Red  training  area. 

The  impending  movement  of  CARGRU-ONE  in  response  to  this  directive  con- 
stitutes a two-fold  decision  problem  to  C0M2ARGRU-0NE  and  his  staff: 
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• Selection  of  the  optimum  operating  area  within  striking  distance 
of  ONRQDA  Island  consistent  with  current  directives  and  the  ex- 
isting situation. 

• Utilizing  the  best  available  and  postulated  information,  selec- 
tion of  the  optimum  transit  route  for  the  movement  to  the  objec- 
tive area. 

2.3  Operations  Area  Selection 

The  Decision  aiding  displays  envisioned  for  assistance  of  the  Task  Force 
Commander  and  his  Staff  in  the  selection  of  an  operating  area  comprises  an 
interactive  system  of  graphic  and  alphanumeric  displays  of  data  which  assists 
them  in  quantizing  the  component  factors  associated  with  the  decision  problem, 
each  as  a function  of  position,  and  in  ordering  their  relative  import.  These 

] I 

j factors  are: 

• Vulnerability /Effectiveness  Summary 

- Weapon  Systems  Effectiveness 

- Vulnerability 
o Air  Threat 
o Surface  Threat 
o Sub- Surface  Threat 

• Logistic  Support 

• Operating  Constraints 

• Aircraft  Route  Planning 

Figure  2-1  is  a depiction  of  the  process  by  which  these  factors  are  used  in 
operations  area  selection,  and  Figures  2-3  to  2-28  illustrate  the  use  of  these 
factors  to  display  an  example  of  ops  area  selection. 

The  analysis  of  COMCARGRU  ONE  and  his  staff  leading  to  the  selection  of  an 
operating  area  is  predicated  on  the  assumption  that  the  most  effective  means 
available  to  accomplish  his  mission  is  to  exploit  his  air  strike  capability, 
either  as  an  immediate  threat  or,  when  directed,  in  projected  operations  against 
Orange  forces  and  installations  on  ONRQDA  Island.  Consequently,  the  selection 
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of  the  area  is  undertaken  in  this  context.  Early  consideration  is  therefore 
given  to  identification  and  analysis  of  targets  in  the  objective  area  to  deter- 
mine the  requirements  for  aircraft  and  weapons.  The  effectiveness  of  the 
available  offensive  assets  as  a function  of  range  is  considered  both  in  terms 
of  weapon  delivery  capability  and  of  target  neutralization  probability.  Vul- 
nerability of  the  Task  Force  to  hostile  air,  surface,  and  sub-surface  threats 
as  functions  of  position  are  then  investigated  singly,  both  in  weapon  delivery 
units  and  probabilistic  terms  of  reference.  The  Commander  is  now  in  a posi- 
tion to  compare  his  force  effectiveness  versus  the  summation  of  his  vulner- 
abilities. An  overly- complicated  display  of  these  numerous  categories  of  in- 
formation is  avoided  by  use  of  a dot  frequency  distribution  pattern  to  portray 
the  positional  distribution  of  a composite  figure- of-merit  for  association 
with  candidate  locations  in  the  operating  theater.  Underlying  digital  data  is 
available  for  specific  quantitative  reference.  The  Commander  next  considers 
the  areas  available  for  support  of  his  operations  by  the  Underway  Replenish- 
ment Group.  Pertinent  to  this  consideration  is  the  radius  of  action  of  Blue 
Air  Force  fighters  based  at  Mid- Ocean  Island,  and  similar  escort  capability 
of  MIG-21  fighters  stages  at  ONRODA.  Before  selecting  his  operation  area, 
the  Commander  reviews  the  existing  operating  constraints  which  will  be  in 
effect  for  his  operations,  and  examines  intelligence  of  early  warning  radar 
coverage  about  ONRODA  to  note  its  effect  on  air  strike  penetration  routes  and 
their  compatibility  with  candidate  positions. 

2.k  Transit  Route  Selection 

Decision  aiding  displays  envisioned  for  utilization  in  transit  route  sel- 
ection comprise  an  interactive  system  of  controls  which  provide  a series  of 
graphic  and  alphanumeric  displays.  Factors  associated  with  this  decision  prob- 
lem which  are  quantized  and  presented  for  evaluation  include: 

• Stipulated  time/distance  capabilities  of  track  assets 

• Current  and  predicted  environmental  conditions  and  their  effects 

• Threat  vulnerability 

- Average  threat  level 

• Route  definition 

• Relative  movement  applications 
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The  process  by  which  these  factors  are  utilized  in  transit  route  selection 
is  shown  in  Figure  2-2. 

Ths  displays  associated  with  quantization  of  effectiveness  and  vulnerabil- 
ity and  consideration  of  logistic  support  and  operating  constraints  which  were 
demonstrated  for  use  in  selection  of  the  operation  area  are  also  available  for 
call-up  in  this  application. 

In  the  example  considered  for  the  ONRQDA  scenario  and  illustrated  in 
Figures  2-29  to  2-46,  the  question  of  route  selection  for  the  carrier  Task 
Force  to  the  selected  operating  position  is  treated  after  selection  of  an 
operations  area.  Initially,  the  curves  representing  furthest-on  positions 
for  12  hour  intervals  at  a selected  30  knot  SOA  are  displayed.  It  is  desir- 
able that  the  hostile  commanders  know  that  the  Task  Force  is  moving  and  that 
the  transit  will  constitute  a threat  to  Orange  forces  on  ONRQDA,  but  not  to 
the  Red  Task  Force.  §y  reference  to  the  predicted  synoptic  weather  chart  the 
Commander  ensures  that  weather  will  be  no  problem  during  the  transit  and  that 
whatever  enemy  surveillance  efforts  are  active  in  the  area  will  not  be  thwarted 
in  detection  of  his  movement.  To  establish  in  his  mind  when  the  Red/Orange 
Commanders  will  certainly  be  apprised  of  his  movement,  he  refers  to  the  dis- 
play of  current/predicted  transits  of  the  Red  surveillance  satellite  COSMOS 
777.  It  appears  that  his  movement  will  be  duly  noted  by  the  0eO947Z  transit 
slightly  North  of  Mid- Ocean  Island.  Reference  is  now  made  to  the  distribu- 
tion of  the  composite  threat  vulnerability  in  the  operating  theater.  The 
operator  having  previously  established  that  the  primary  threat  to  the  Task 
Force  at  the  selected  operating  position  is  the  air  threat  from  the  direction 
of  ONRQDA  Island,  an  intermediate  transit  point  is  designated  to  minimize  the 
vulnerability  to  this  threat  during  final  approach  to  the  operating  position. 
Another  intermediate  transit  point  is  selected  to  clear  North  Cape  on  Mid- 
Ocean  Island  by  50  N Mi,  and  a third  is  established  as  a loiter  point  for  con- 
duct of  local  operations  en  route.  The  most  direct  route  is  generated  as  de- 
fined by  these  points.  The  total  distance  is  calculated  and  displayed  to  be 
1320  N Mi.  The  distribution  of  hostile  threat  level  along  the  tentative  track 
is  ascertained. 
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A significant  perturbation  of  the  Transit  Route  Selection  problem  is 
occasioned  by  receipt  of  intelligence  apprising  the  Task  Group  Conmander  of 
the  establishment  of  an  Orange  submarine  patrol  area  to  the  Eastward  of  Mid- 
Ocean  Island.  The  immediate  effects  of  this  information  are  two-fold.  First, 
of  course,  it  fills  the  blank  portion  of  the  available  intelligence  concern- 
ing the  whereabouts  and  employment  Of  the  one  Orange  operational  submarine. 
Secondly,  and  more  immediately,  it  defines  a point  threat  to  be  neutralized 
or  avoided  in  the  transit  of  CARGKU  ONE  to  the  selected  operating  area.  In 
view  of  the  fluidity  of  the  general  situation  in  the  ONRGDA  theater,  it  is 
deemed  advisable  to  avoid  the  submarine's  100  N Mi  weapon  range  by  end- run 
to  the  northward.  (Mere  avoidance  of  the  submarine's  detection  range  is 
considered  insufficient  in  view  of  the  fact  that  the  location  of  a Red  AGI, 
which  is  known  to  be  in  the  operating  theater,  has  been  unknown  for  an  ex- 
tended period).  Accordingly,  a second  transit  route  which  diverts  the  track 
to  the  northward  sufficiently  to  ensure  a CPA  of  120  N Mi  is  defined. 

2.5  Use  of  Data  Displays 

In  the  two  illustrative  examples  of  decision  aiding  techniques  des- 
cribed in  this  report  the  user  is  provided  with  a series  of  visual  aids  that 
he  may  call  up  as  desired  to  assist  him  in  formulating  his  ultimate  decision. 
These  consist  of  interactive  alphanumeric  and  graphic  displays  in  which  data 
germane  to  the  problem  at  hand  has  been  retrieved  upon  command,  summarized 
and  formatted  for  quick,  precise  assimilation  by  the  user.  Included,  as  well, 
is  the  capability  for  the  user  to  exercise  trial  solutions  and  analyze  their 
outcome . 

At  the  start  of  the  sequence  the  user  is  presented  an  alphanumeric  dis- 
play which  lists  the  categories  of  data  displays  he  may  exercise  to  aid  Mm 
in  arriving  at  his  solution,  and  the  user  is  requested  to  enter  through  a key- 
board the  category  he  wishes  to  examine.  As  each  category  is  called  up  by 
the  user,  and  they  may  be  called  up  in  any  order  and  at  different  levels  of 
detail,  the  user  is  presented  with  further  prompts  or  requests  to  select  op- 
tions from  a displayed  menu. 
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A way  of  summarizing  numerical  data  associated  with  specific  geograph- 
ical areas  is  the  dot  grid  in  which  the  density  of  dots  within  an  area  is 
proportional  to  the  numerical  value  to  be  represented.  This  type  of  data 
display  expresses  the  results  in  a clear,  concise  visual  form  that  can  quickly 
be  interpreted  by  the  user.  In  order  to  apply  the  dot  grid  display,  the  geo- 
graphic area  under  consideration  is  divided  into  equal  squares.  The  number 
of  dots  in  each  square  is  proportional  to  the  value  of  the  parameter  being 
considered,  or  represents  a figure  of  merit  for  the  area  within  the  square. 

For  example  the  dots  within  a square  may  be  proportional  to  the  tons  of  bombs 
that  can  be  delivered  per  day  against  ONRODA  airfield  if  the  Task  Force  were 
stationed  within  that  square. 

Since  the  two  decision  problems  considered  are  geographical  or  area  or- 
iented, many  of  the  data  displays  consist  of  graphical  data  overlayed  on  a map 
display  which  is  presented  on  the  left  screen  with  supplementing  alphanumeric 
data  on  the  right  screen. 

Although  data  can  be  entered  through  a keyboard,  the  use  of  a light  pen 
type  of  device  that  would  permit  the  user  to  trace  a tentative  route  on  the  map 
display  would  permit  rapid  examination  of  alternate  courses. 

Of  additional  interest  to  the  tactical  commander  is  the  facility  to 
solve  and  display  the  results  of  relative  motion  vector  problems.  A comput- 
erized maneuvering  board  type  of  presentation  was  employed  which  provides  the 
user  with  immediate  comprehension  of  the  relative  motion  involved  in  hypothe- 
sized situations  and  gives  a visual  description  of  the  associated  time,  dis- 
tance and  speed  parameters. 

Although  these  data  displays  present  computer  processed  data  and  calcu- 
lated results  in  an  easy  to  assimulate  visual  foim,  they  do  not  foimulate  any 
judgement  decisions.  The  user  controls  the  amount  of  computer  aiding  he  wish- 
es, selects  the  input  data,  and  foxms  his  own  decisions.  The  data  displays 
can  however  greatly  reduce  the  effort  that  would  normally  be  required  by  him 
for  data  retrieval,  organization,  condensation,  and  assimilation.  They  also 
serve  as  a check  list  to  assure  he  explores  the  major  facets  pertaining  to 
the  decision  making  process. 
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2.6  Example  of  Best  Operation  Area  Selection 


Presented  in  this  section  is  a step-by-step  example  of  how  the  data  dis- 
plays previously  discussed  would  be  used  by  the  Task  Force  Commander  and  his 
staff  to  aid  in  the  selection  of  the  best  area  to  station  their  Task  Force 
for  the  purpose  of  carrying  out  the  desired  mission.  A series  of  graphic  and 
alphanumeric  displays  are  presented  in  the  same  order  they  would  noimally  be 
called  up  by  the  user.  The  displays  which  would  appear  on  the  left  screen 
are  shown  on  the  left  pages  and  right  screen  presentations  on  right  pages. 

The  operational  use  of  each  set  of  displays  is  explained  by  the  text  located 
under  the  set  of  displays. 
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FIGURE  2-3  TARGET  AREA  MAP 

The  user,  having  selected  the  Best  Operations  Area  Mode,  is  shown  Figure 
2-4  - Mission  Parameters  on  the  right  hand  screen,  in  which  he  is  requested  to 
designate  three  mission  related  parameters: 

• Primary  Mission  Objectives  - enters  mission  number  (M) 

• Target  Area  - enters  zone  number  (Z) 

• Mission  Time  - enters  start  and  duration  (S  and  D). 

After  entry  of  these  mission  parameters  Figure  2-3,  Target  Area  Map,  is  dis- 
played on  the  left  hand  screen  for  reference  purposes.  The  Target  Area  Map 
is  a fixed  display  which  shows  the  areas  occupied  by  Grey,  Yellow  and  Orange 
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FIGURE  2-k  MISSION  PARAMETERS 

forces  and  location  of  key  strategic  installations.  This  map  may  be  called  up 
at  any  future  time  by  the  user  for  reference.  The  user  would  next  activate 
the  CONTINUE  mode  and  the  following  displays  would  appear. 
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FIGURE  2-5  TARGET  AREA  MAP 

Listed  in  Figure  2-6  - Ops  Areas  Factors  are  8 factors  the  user  may  wish  to 
examine  before  selecting  an  operating  area  from  which  to  conduct  the  air  strikes. 
All  eight  factors  would  normally  be  selected  one  at  a time  and  In  order  by  the 
user.  However,  when  short  of  time,  the  user  may  select  only  those  factors  he 
wishes  to  examine.  His  selections  should  include  Factor  1 - WEAPON  SYSTEM 
EFFECTIVENESS,  as  this  is  an  interactive  display  in  which  he  sets  the  target  and 
weapon  mix  criteria.  Once  the  user  responds  to  the  questions  asked  in  Factor  1, 
he  may  proceed  directly  to  Factor  5 which  summarizes  the  results  of  Factors  1 
through  4 and  shows  a dot  grid  presentation  indicating  those  geographical 
operating  areas  In  which  the  vulnerability/effectiveness  ratio  Is  minimized. 
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FIGURE  2-6  OPS  AREA  FACTORS 

Factors  6,  7,  and  8 are  not  Interactive  displays,  and  selecting  these  factors  re- 
sults in  pertinent  information  being  displayed  in  alphanumeric  tables  and  graphic 
formats  for  the  user's  examination.  Die  normal  procedure  of  calling  up  *n  the 
factors  in  order  is  demonstrated  in  this  text.  Figure  2-5,  which  is  the  same  as 
Figure  2-3,  remains  on  the  left  screen  until  the  user  selects  Factor  1 - WEAPON 
SY8HM8  EFFECTIVENESS  at  which  time  the  displays  shown  on  the  next  two  pages 
appear. 
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FIGURE  2-7  TARGETS  - ONRODA  ISLAND 


The  user,  having  selected  WEAPON  SYSTEMS  EFFECTIVENESS  as  the  Ops  Area 
Factor,  would  be  presented  with  an  interactive  display.  Figure  2-8  - Air 
Strike  Targets  and  Weapons,  on  the  right  and  Figure  2-7  - Targets  - 
Island  on  the  left.  ONRODA  Island  has  been  divided  into  four  zones  A,  B,  C 
D as  indicated  on  Figure  2-7.  The  precise  coordinates  of  the  targets  are  known, 
of  course,  however,  for  the  purpose  of  the  forthcoming  display  presentations 
it  suffices  to  group  the  target  locations  into  four  quadrants  in  order  to 
simplify  the  computations  that  would  be  required  if  the  system  was  actually 
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FIGURE  2-8  AIR  STRIKE  TARGETS  AND  WEAPONS 

Implemented.  The  user  selects  a target  by  entering  the  "T"  number.  He  selects 
the  aircraft  he  wishes  to  deploy  and  the  weapon  mix  by  entering  the  "W”  number 
and  the  appropriate  two  digit  number.  The  computer  now  has  enough  information 
to  detemine  either  (Pi)  the  kllotons  per  day  that  could  be  dropped  on  the  se- 
lected target  with  the  selected  aircraft  and  weapon  mix  as  a function  of  the 
range  from  target,  or  (P2)  probability  of  neutralizing  the  target  within  a time 
span  of  24  hours  with  the  selected  aircraft  and  weapon  mix  as  a function  of  the 
range  from  the  target.  The  user  selects  either  (EL)  or  (P2)  or  both. 


If  the  user  selects  kilotons/day  then  Figure  2-9  would  appear  on  the 
left  screen  and  Figure  2-8  shown  on  the  previous  page  would  remain  on  the 
right  screen.  Figure  2-9  is  computed  based  on  one -half  of  the  selected  air- 
craft flying  each  sortie.  The  weight  of  the  fuel  required  for  the  aircraft 
to  reach  the  target,  perform  the  air  strike  and  return  plus  the  weight  of 
required  reserve  fuel  is  subtracted  from  the  aircraft's  total  payload  capacity 
for  each  range  increment  when  determining  weapon  delivery  capacity.  If  the 
user  selects  PROBABILITY  OF  NEUTRALIZATION,  then  Figure  2-10  would  appear  on 
the  left  screen  and  Figure  2-8  remain  on  the  right  screen.  Figure  2-8  remins 
Interactive  enabling  the  user  to  try  other  selections  and  view  the  correspond!  ig 
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FIGURE  2-10  PROBABILITY  OF  NEUTRALIZING  TARGET 


results  on  the  left  screen.  If  the  user  elects  to  view  both  the  kilotons  and 
probability  presentations  simultaneously.  Figure  2-9  is  shown  on  the  left  screen 
and  Figure  2-10  on  the  right.  PROBABILITY  OF  NEUTRALIZING  THE  TARGET  is 
normally  a time  dependent  function  (i.e.,  number  of  sorties  flown).  In  this 
presentation  the  probability  of  neutralization  is  computed  by  allotting  tho 
payload  that  can  be  delivered  within  2k  hours  (which  is  a function  of  the  range 
from  Ops  Area  to  target)  equally  among  the  selected  weapons.  At  this  time, 
probability  of  neutralization  is  computed  without  regard  to  enemy  defenses. 
PROBABILITY  OF  SURVIVAL  (VULVERABILITY)  is  introduced  next.  Figures  2-9  and 
2-10  allow  the  user  to  examine  where  he  should  station  his  Task  Force  to  achieve 
relative  degrees  of  effectiveness  while  operating  unopposed.  He  may  bypass 
these  displays  and  elect  to  go  directly  to  the  combined  Vulnerability-Effective- 
ness displays  (Figures  2-21  and  2-22). 
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FIGURE  2-11  AOB  MAP  DISPLAY 


Having  examined  own  force  effectiveness  as  a function  of  Ops  Area  location 
the  user  can  now  explore  own  force  vulnerability  as  a function  of  Ops  Area 
location. 


Vulnerability  to  air,  surface  and  sub- stir face  threat  can  be  examined 
separately.  The  user  would  again  call  up  Figure  2-6  - Ops  Area  Factors  and 
select  Option  2 - VULNERABILITY  TO  AIR.  Figure  2-12  - Air  Order  of  Battle  (AOB) 
would  appear  on  the  right  screen  and  Figure  2-11  - AOB  Map  Display  on  the  left. 
These  displays  allow  the  user  to  analyze  the  AOB  situation.  Other  data  such  as 
enemy  aircraft  combat  radius  for  various  weapon  loads,  although  not  shown  in 
this  test,  can  also  be  called  up  by  the  user  for  review.  The  ONRQDA  scenario 
dictates  that  no  military  aircraft  are  currently  based  on  ONRODA  Island. 
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FIGURE  2-12  ENEMY  AIR  ORDER  OF  BATTLE 
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However,  fighters  could  be  staged  there  to  provide  escort  for  the  Badger  and 
Beagle  bombers.  The  maximum  ranges  used  in  the  following  delivery  capability 
displays  are  based  on  the  maximum  escort  range  of  MIG-21  fighters  staged  at 
ONRODA  Island.  The  user  can  now  select  the  type  of  presentation  desired: 

PI  - Kilotons  of  weapons  the  enemy  aircraft  could  deliver  on  the  Task 
Force  per  day  as  a function  of  task  force  location,  or 
P2  - Probability  of  neutralization,  or  both 


FIGURE  2-13  ORANGE  AIR  THREAT  - KILOTONS/DAY 

If  the  user  selects  kilotons/day,  then  Figure  2-13  appears  on  the  left 
screen  and  Figure  2-12  shown  on  the  previous  page  would  remain  on  the  right 
screen.  Figure  2-13  is  based  on  one-half  of  the  aircraft  stationed  at  each 
airfield  (HA-1  and  HA-2)  making  each  sortie.  The  dotted  curve  represents 
the  maximum  escort  range  of  MIG-21s  that  were  refueled  at  QNRQDA  Island, 
and  represents  the  cut-off  point  beyond  which  bombers  would  not  penetrate 
without  fighter  escort.  As  previously,  delivery  capacity  is  payload  minus 
fuel  for  a specific  range.  If  the  user  selected  PROBABILITY  OF  ORANGE 
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FIGURE  2-14  PROB.  OF  ORANGE  NEUTRALIZING  TASK  FORCE  - AIR 

NEUTRALIZING  OWN  TASK  FORCE,  then  Figure  2-14  would  appear  on  the  left  screen 
and  Figure  2-12  remain  on  the  right.  If  both  Figures  2-13  and  2-14  are 
selected  simultaneously  they  would  appear  as  shown  above.  Hostile  effectiveness 
in  both  figures  is  computed  without  regard  to  own  Task  Force  defense  capability. 


FIGURE  2-15  ENEMY  MISSILE  DELIVERY  CAPABILITY 

The  user  would  now  proceed  to  examine  the  Task  Force  vulnerability  to 
surface  threat.  This  would  be  done  by  calling  up  Figure  2-6  - Ops  Area  Factors 
and  selecting  Factor  3 - VULNERABILITY  TO  SURFACE  THREAT.  Figure  2-l6  - 
Surface  Threats  would  then  be  displayed  on  the  Right  Screen.  This  display 
enables  the  user  to  examine  the  location  and  quantity  of  surface  threats.  He 
then  selects  whether  he  wishes  to  display  "current  status"  or  "furthest  on". 
Current  status f which  is  called  by  entering  T=0,  displays  the  missile  coverage 
as  shown  in  Figure  2-15  - Enemy  Missile  Delivery  Capability. 
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FIGURE  2-16  SURFACE  THREATS 


The  numeric  character  within  each  range  circle  on  Figure  2-15  represents  the 
number  of  missiles  that  can  reach  any  spot  within  that  circle  if  all  enemy 
missile  carrying  ships  launch  their  full  deckload.  The  position  of  the  enemy 
task  force  is  designated  by  code  numbers  (i.e.,  HS-3)  displayed  by  the  center 
of  the  circle. 
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FIGURE  2-17  ENEMY  MISSILE  DELIVERY  CAPABILITY  - 12  HRS 

If  the  user  wishes  to  examine  what  the  Task  Force  vulnerability  to  surface 
threats  would  be  at  some  future  time,  he  may  specify  the  time  increment  from 
present  to  desired  future  time  in  hours  (Display  "Furthest  On"  entry  in  Figure  2-l6). 
The  distance  each  surface  threat  could  travel  in  the  specified  time  increment  is 
then  computed  and  added  to  its  missile  range.  Using  12  hours  as  an  example,  the 
missile  coverage  would  then  be  displayed  as  shown  in  Figure  2-17  - Enemy  Missile 
Delivery  Capability  - 12  Hrs.  This  represents  a worst  case  condition,  since  it 
shows  the  maximum  number  of  missiles  that  could  be  brought  to  bear  upon  a Task 
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FIGURE  2-18  PROB.  OF  ORANGE  NEUTRALIZING  TASK  FORCE  - SURFACE 


Force  in  each  area  if  all  missile  carrying  ships  steamed  toward  that  area 
for  12  hours  and  launched  their  complete  deck  load  of  missiles  at  the  Task  Force. 
If  desired,  the  user  may  display  enemy  surface  threat  capability  as  a function 
of  probability  as  shown  in  Figure  2-l8  - Probability  Of  Orange  Neutralizing 
Task  Force-Surface,  by  selecting  the  P2  Option  shown  in  Figure  2-16.  Probability 
of  neutralization  may  be  displayed  for  either  "Current  Status"  or  "Furthest  On*. 
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FIGURE  2-19  ENEMY  SUB  LAUNCHED  MISSILE  COVERA® 

The  user  would  now  proceed  to  examine  task  Force  vulnerability  to 
sub-surface  threats  by  calling  up  Figure  2-6  - Ops  Area  Factors  and  selecting 
Factor  4 - VULNERABILITY  TO  SUB-SURFACE  THREATS.  Figure  2-20  - Sub-Surface 
Threats  would  then  be  displayed  on  the  right  screen.  The  user  would  select 
either  "CURRENT  STATUS"  or  "FURTHEST  ON"  and  specify  time  as  he  did  when 
analyzing  surface  threats.  The  user  would  also  select  type  of  presentation. 

If  he  selected  PI  - MISSILE  COVERAGE  and  CURRENT  STATUS,  Figure  2-19  - 
Eneny  Sub  Launched  Missile  Coverage  would  appear  on  the  left  screen.  Unless 
the  submarine  is  in  port  its  precise  location  is  generally  not  known.  The 
user  viewing  Figures  2-19  and  2-20  can  see  the  submarine's  approximate  position 
at  the  time  of  its  last  contact.  If  he  requests  FURTHEST  ON  (say  after 
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FIGURE  2-20  SUB-SURFACE  THREATS 

12  hours)  the  distance  the  submarine  can  travel  at  submerged  running  speed 
for  that  time  is  added  to  the  missile  range  and  a display  similar  to  Figure  2-17 
far  surface  threats  would  result.  As  in  the  surface  threat  case,  the 
probability  of  the  Orange  submarines  neutralizing  the  Blue  Task  Force  may  be 
displayed  for  either  "CURRENT  STATUS"  or  "FURTHEST  ON".  The  effects  of 
torpedoes  were  not  included  as  the  torpedo  range  circles  that  would  have  to 
be  be  added  would  be  too  snail  to  be  noticed  on  the  display  for  the  "CURRENT 
STATUS"  case.  However,  their  effects  could  be  included  in  the  "FURTHEST  (Mr" 
case  by  letting  their  coverage  area  be  equal  to  the  submarines  cruising  range 
during  the  specified  time  Interval. 
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FIGURE  2-21  DOT  GRID  - VULNERABILITY-EFFECTIVENESS  SUMMARY 


The  user  has  now  examined  own  force  effectiveness  against  a selected 
target  and  own  force  vulnerability  to  air,  surface  and  sub-surface  threats, 
all  as  a function  of  own  force  position  for  given  orders  of  battle.  He  is 
now  in  a position  to  try  and  combine  all  this  data  somehow  to  determine  the 
best  operating  area  for  his  task  force  based  on  generated  effectiveness  and 
vulnerability  information.  This  is  where  the  dot  grid  presentation  of 
computer  generated  results  can  serve  as  an  effective  aid  for  the  user.  The 
user  would  once  again  call  up  Figure  2-6  - Ops  Area  Factors  and  select 
Factor  5 - VULNERABILITY/EFFECTIVENESS  SUMMARY.  Figure  2-21  - Dot  Grid  - 
Vulnerability-Effectiveness  Summary,  without  the  dots  would  appear  on  the  left 
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FIGUKE  2-22  VULNERABILITY  - EFFECTIVENESS  SUMMARY 


screen  and  Figure  2-22  - Vulnerability  - Effectiveness  Summary  without  numbers 
would  appear  on  the  right  screen.  The  user  would  specify  the  boundaries  of  the 
dot  grid  presentation  by  identifying  the  rows  and  columns  (i.e.,  B-N,  3-10), 
causing  the  dots  in  Figure  2-21  and  numbers  in  Figure  2-22  to  be  filled  in. 

The  less  dots  within  a grid  square  on  Figure  2-21,  the  better  it  is  as  an 
operating  area  for  the  Task  Force  considering  vulnerability  and  effectiveness. 
The  best  areas,  in  order,  are  identified  with  corresponding  data  in  Figure  2-22 
The  numbers  in  Figure  2-22  represent  the  probabilities  previous  discussed, 
except  for  FOM  (Figure  of  Merit)  which  is  a normalized  relative  rating. 


Once  the  user  has  analyzed  the  VULNERABILITY  - EFFECTIVENESS  SUMMARY 
he  may  wish  to  examine  in  which  areas  he  can  be  best  supported  by  Underway 
Replenishment  Group  (URG).  This  is  done  by  calling  up  Figure  2-6  and 
selecting  Factor  6 - LOGISTIC  SUPPORT.  Figure  2-24  - Radii  Of  Action  and 
Figure  2-23  URG  Rendezvous  Areas  are  displayed  as  shown.  Figure  2-23  shows 
the  maxi  mum  distance  the  MIG-21  fighters  could  escort  the  Orange  bombers 
(i.e..  Badgers).  This  envelope  is  shown  as  boundary  line  A and  it  may  be 
assumed  that  areas  to  the  left  of  line  A would  be  safe  from  attack  by  orange 
bombers.  Line  B in  Figure  2-23  shows  the  maximum  escort  range  of  blue  F-4 
fighters  based  at  Mid  Ocean  Island  (MOI).  URG  ships  operating  to  the  right 
of  boundary  line  B would  have  no  MOI  based  fighter  protection. 
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- FIGURE  2-24  RADII  OF  ACTION 

Figure  2-24  lists  the  meaning  of  each  boundary  line.  Other  boundary  lines 
such  as  the  radius  of  action  of  patrol  boats  based  at  specific  ports  can  be 
included  on  the  displays.  Time  related  information  can  also  be  presented. 
For  example,  if  the  Task  Force  wished  to  vacate  its  operating  area  for  no 
more  than  12  hours  for  replenishment  purposes,  range  circles  could  be  drawn 
about  the  Task  Force  which  represent  the  maximum  distance  it  could  travel, 
take  on  fuel  and  supplies,  and  return  in  time.  The  distance  Orange  surface 
forces  could  travel  in  the  same  12  hours  could  also  be  shown. 
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FIGURE  2-25  BOUNDARY  CONSTRAINTS 


Hie  user  may  wish  to  examine  operating  constraints.  This  is  done  by- 
calling  up  Figure  2-6  and  selecting  Factor  7 - OPERATING  CONSTRAINTS.  The 
information  is  displayed  by  Figure  2-25  - Boundary  Constraints  on  the  left 
screen  and  Figure  2-26  - Operating  Constraints  on  the  right  screen. 

Figure  2-25  is  a map  display  with  prohibited  areas  and  areas  of  special 
interested  overlayed.  Figure  2-26  defines  the  significance  of  the  areas 
and  the  operating  rules  regarding  them.  Figure  2-26  also  can  present  rules 
of  engagement,  defense  measures,  sanctuary  areas,  general  guidance  and 
information  pertinent  to  selecting  an  operating  area.  The  information 
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FIGURE  2-26  OPERATING  CONSTRAINTS 


contained  in  the  map  display  (Figure  2-25)  could  also  be  shown  using  a dot  grid 
presentation  similar  to  Figure  2-21.  Circular  areas  however,  would  have  to  be 
approximated  by  the  straight  line  boundaries  of  the  grid  squares.  Variations 
in  the  density  of  the  dots  could  define  different  areas  as  they  overlapped. 

The  information  shown  on  Figure  2-25  could  be  combined  with  the  information 
shown  on  Figure  2-21  in  a dot  grid  presentation.  If  this  were  done,  it  is 
suggested  all  prohibited  areas  be  blacked  in  with  the  maximum  dot  density 
pattern. 
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The  user  would  now  proceed  to  examine  which  Ops  areas  would  provide  access 
to  favorable  air  strike  penetration  routes.  This  is  done  by  calling  up  Figure  2-6 
again  and  selecting  Factor  8 - AIRCRAFT  RCKJTE  PLANNING.  Figure  2-27  and  2-28 
would  then  appear  on  the  left  and  right  screens  respectively.  The  outer  dotted 
boundary  line  on  Figure  2-27  shows  the  combined  radar  line  of  sight  (LOS)  de- 
tection ranges  for  all  Early  Warning  (EW)  radars  located  on  QNRQDA  Island  that 
would  apply  to  aircraft  trying  to  penetrate  at  an  altitude  of  1000  feet  above 
sea  level.  The  inner  dotted  boundary  shows  the  same  information  for  a 
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FIGURE  2-28  EW  RADAR  DATA 
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500  foot  penetration  altitude . Figure  2-28  tabulates  the  radar  data  used  in 
generating  the  curves  in  Figure  2-27.  The  radar  horizon  was  calculated  by 
considering  the  elevation  of  the  radar  site,  the  radar  antenna  structure 
height  and  the  aircraft  penetration  altitude . The  radar  horizons  axe  shown 
in  Figure  2-28  for  both  the  500  and  1000  ft.  aircraft  penetration  altitudes. 

It  may  be  noted  the  radar  at  location  D is  aboard  a dockside  ship.  These  two 
displays  complete  the  presentations  that  may  be  called  up  by  the  user.  He 
may  go  back  and  display  any  previous  ones  or  change  any  data. 

I 


The  user,  having  exercised  the  available  computer  aided  display  capabil- 
ities by  examining  the  eight  Ops  Area  Factors,  is  nowin  a position  to  make 
a decision.  Some  of  the  data  displays  he  has  examined  point  him  toward  one 
or  more  specific  areas  as  being  the  best  choices  for  the  specific  factors  con- 
sidered. However  an  attempt  is  not  made  to  combine,  through  the  computerized 
mechanization,  the  results  from  all  eight  factors  considered.  The  relative 
importance  of  each  factor  (e.g. , weapon  system  effectiveness,  vulnerability, 
logistic  support)  in  formulating  the  final  decision  must  be  left  to  the  user. 


2.7  Example  of  Transit  Route  Selection 

Presented  in  this  section  is  a step-by-step  example  of  how  the  data  dis- 
play would  be  used  by  the  Task  Force  Commander  and  his  staff  to  aid  in  the 
selection  of  the  Best  Transit  Route.  The  format  is  the  same  as  was  used  in 
the  preceding  example  of  Best  Operation  Area  Selection. 
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FIGURE  2-29  THEATER  MAP 

The  user,  having  selected  the  Transit  Route  Mode,  is  shown  Figure  2-30  - 
Transit  Route  Select  Menu  on  the  right  screen  in  which  he  is  requested  to 
select  the  categories  he  wishes  to  examine.  The  user  would  normally  call  up 
and  examine  all  seven  categories  shown  in  Figure  2-30,  one  at  a time  and  in 
the  listed  order,  although  he  has  the  option  of  not  calling  up  an  the  categories. 
On  the  left  screen  Figure  2-29  - Theater  Map  is  displayed,  which  shows  a 
graphical  map  picture  of  the  mission  area  large  enough  to  include  his  present 
location  and  all  possible  destinations  for  the  previously  established  mission. 
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FIGURE  2-30  TRANSIT  ROUTE  SELECT  MENU 


As  each  category  is  called  by  the  user  a series  of  displays  will  be  shown,  some 
of  which  request  more  information,  allowing  the  user  to  peruse  formatted  data 
germane  to  his  decision  making  process  and  simulate  certain  trial  conditions. 

In  order  to  demonstrate  the  full  capability  available  to  the  user,  the  dis- 
plays associated  with  all  seven  categories  will  be  shown  on  the  following  pages 
in  the  order  listed  in  Figure  2-30. 
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FIGURE  2-31  TF  FURTHEST-ON  POSITIONS 


The  user,  having  selected  the  TIME  DISTANCE  CAPABILITY  category,  would  be 
presented  with  an  interactive  display,  Figure  2-32  - Time/Distance  Capability 
Prompt,  which  requests  further  information.  The  user  enters  the  present  position 
and  speed  of  advance  of  the  Task  Force,  the  time  increment  for  which  the 


extrapolated  positions  are  to  be  calculated  and  the  length  of  the  time  (time 
period)  over  which  the  extrapolations  are  to  extend.  The  extrapolated  or  Furthest- 
On  positions  that  could  be  reached  by  the  Task  Force  are  then  shown  in  Figure  2-31  - 
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FIGURE  2-32  TIME/DISTANCE  CAPABILITY  PROMPT 

TF  Furthest-On  Positions  on  the  left  screen  as  an  overlay  of  the  Theater  Map. 
Included  in  Figure  2-31  are  the  boundary  constraints  previously  established 
under  OPERATING  CONSTRAINTS  while  selecting  the  Best  Operating  Area.  For  the 
example  shown  in  Figure  2-31,  the  time  increment  is  12  hours  and  the  time 
period  is  48  hours.  The  curves  represent  the  locus  of  points  that  could  be 
reached  by  the  Task  Force  in  each  12  hour  increment  and  gives  the  user  an 
appreciation  of  his  time-distance  capability.  The  effects  of  altering  course 
to  avoid  land  mass,  shoals  and  prohibited  areas  are  included. 
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FIGURE  2-33  SYNOPTIC  WEATHER  CHART 


The  user,  having  examined  the  Task  Force  Time  Distance  Capability,  can 
now  explore  the  effects  of  environment  on  his  transit  route  selection.  He 
would  call  up  Figure  2-30  and  select  C2  - ENVIRONMENT.  Figure  2-34  - Environ- 
ment Menu  would  be  displayed  on  the  right  screen.  The  user  would  select  the 
desired  environment  feature  and  it  would  be  graphically  displayed  on  the  left 
screen.  The  area  weather  can  be  displayed  either  as  a conventional  synoptic 
weather  chart  as  shown  in  Figure  2-33  or,  if  information  is  available,  as  a dot 
grid  presentation  where  the  density  of  the  dots  corresponds  to  wind,  pre- 
cipitation or  sea  state.  The  operator  must  also  designate  from  the  menu  in 
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FIGURE  2-34  ENVIRONMENT  MENU 

Figure  2-34  the  Time  of  Interest  (i.e.,  weather  now  or  that  predicted  "X"  hours 
from  now),  and  dot  grid  or  chart  presentation.  Other  environment  parameters 
the  user  may  select  and  observe  on  the  left  screen  are : Hydrography  - depth 

contour  lines,  shoals  - and  sonar  detection  parameters . The  selected  environ- 
mental parameter  is  always  overlayed  on  the  same  map  background  in  order  to 
constrain  the  software  requirements. 


a 


2-45 

- — ■■  — — J 


FIGURE  2-35  HOSTILE  SATELLITE  COVERAGE 

In  order  to  proceed  in  his  examination  of  parameters  affecting  the  transit 
route  selection,  the  user  would  again  call  up  Figure  2-30  and  select  C3  - 
SURVEILLANCE  CAPABILITY.  The  resulting  displays  would  show  the  coverage  of 
hostile  surveillance  satellites  as  a function  of  time  superimposed  on  the 
theater  map  background.  Examination  of  these  displays  would  permit  the  user 
to  plan  his  transit  route  to  either  coincide  with  the  satellite  coverage  if 
he  wishes  to  announce  his  intentions  or  temporarily  avoid  the  satellite  coverage 
if  he  wishes  to  operate  covertly  as  long  as  possible. 

In  the  context  of  the  ONRGDA  theater  situation  it  is  desirable  that  the 
Orange  and  Red  authorities  cure  apprised  that  the  Task  Force  is  moving  and  will 
constitute  a threat  to  the  Oranges  Forces  on  ONRODA,  but  not  to  the  Red  Task  Force. 
The  achievement  of  this  goal  would  be  enhanced  by  planning  the  transit  route  to 
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FIGURE  2-36 


HOSTILE  SATELLITE  SURVEILLANCE  PROMPT 
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coincide  with  the  satellite  coverage  as  soon  as  possible.  Figure  2-36  - Hostile 
Satellite  Serve illance  Prompt  would  be  displayed  on  the  right  screen,  which  re- 
quests the  user  to  identify  the  beginning  and  end  time  of  the  period  for  which  he 
wishes  to  see  the  satellite  coverage  and  to  identify  the  satellite.  The  user 
also  enters  his  present  position,  course  and  speed.  When  the  user  makes  his 
selections  called  for  in  Figure  2-36  a graphical  display  Figure  2-35  - Hostile 
Satellite  Coverage  appears  on  the  left  screen.  This  figure  shews  the  satellite 
coverage  and  the  user's  dead  reckoned  position  at  each  satellite  pass.  In  the 
example  shown  the  beginning  time  corresponded  to  0500  hours  and  the  end  time 
0200  hours  the  next  day.  Satellite  passes  occurred  at  the  times  associated  with 
positions  1,  2,  3 and  4 as  shown  in  Figure  2-35.  It  can  be  noted  the  user  canes 
under  the  satellite  surveillance  coverage  at  position  4 and  is  noted  in  Figure 
2-36. 


FIGURE  2-37  COMPOSITE  THREAT  VULNERABILITY 

The  user  would  now  examine  how  vulnerable  his  Task  Force  would  be  to  enemy 
threats  as  he  passed  through  various  geographical  areas.  This  is  done  by 
calling  up  Figure  2-30  and  selecting  C-4  VULNERABILITY.  Figure  2-38  - Threat 
Vulnerability  Menu  would  appear  on  the  right  screen  in  which  the  user  is  re- 
quested to  select  the  type  of  threat:  air,  surface,  sub-surface  or  composite 

(all  threats)  and  the  time  for  which  the  threat  levels  are  to  be  calculated. 

The  vulnerabilities  would  be  calculated  and  displayed  as  shown  in  Figures  2-11 
through  2-22  in  the  Ops  Area  Selection  section.  For  example  air  vulnerability 
can  be  expressed  as  kilotons  per  day  that  can  be  delivered  in  specific  geo- 
graphic areas,  or  as  probability  of  the  Task  Force  being  neutralized  by 
Orange  Force  in  specific  geographical  areas.  Surface/sub-surface  vulnerability 
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FIGURE  2-38  THREAT  VULNERABILITY  MENU 


can  be  displayed  as  missile  delivery  capability  or  as  probability  of  neutrali- 
zation. The  distance  the  surface  and  sub-surface  threats  can  travel  until  the 
specified  time  is  reached  is  also  included  in  their  weapon  coverage  as  in  the 
Ops  Area  selection  case.  If  the  user  desires  to  go  directly  to  the  total  com- 
posite effect  of  considering  all  threats  by  choosing  v4  - COMPOSITE  in  Figure 
2-38,  a dot  grid  presentation  would  be  displayed  on  the  left  screen  as  shown 
in  Figure  2-37  - Composite  Threat  Vulnerability.  The  user  would  try  to  select 
a course  that  passes  through  the  areas  of  least  dot  density  when  considering 
own  Task  Force  vulnerability. 


FIGURE  2-39  TRANSIT  AREA  CHART 


The  user  may  now  wish  to  examine  the  time-distance  relationships  that 
would  result  from  certain  selected  transit  routes.  The  user  would  r^n  up 
Figure  2-30  and  select  C5  - ROUTE  DEFINITION.  Figure  2-40  - Route  Definition 
Parameters  would  then  be  displayed  on  the  right  screen  and  Figure  2-39  - 
Transit  Area  Chart  on  the  left  screen.  The  user  would  enter  the  latitude  and 
longitude  of  his  start  point,  up  to  nine  way  points  (intermediate  points 
to  be  passed  through  along  the  way),  and  his  final  destination.  An 
alternate  way  of  describing  the  transit  route  would  be  "hooking"  all 
the  points  with  a light  pen  or  similar  device  on  Figure  2-39.  The  speed  of 


FIGURE  2-40  ROUTE  DEFINITION  PARAMETERS 


advance  between  each  two  points  may  be  entered,  or,  if  the  speed  is  to  remain 
constant  over  several  legs,  only  the  beginning  and  end  points  over  which  the 
speed  is  to  remain  constant  is  entered  (i.e.,  S6,  7).  If  a rendezvous  is  to 
occur  or  there  is  a requirement  to  remain  at  a way  point  for  a period  of  time 
(e.g.,  take  on  fuel)  this  data  is  entered  by  identifying  the  way  point  and 
entering  the  hours  to  remain  on  station,  (i.e.,  T6  - 14)  for  each  applicable 
way  point. 
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FIGURE  2-41  TRANSIT  AREA  ROUTE  CHART 


Upon  establishing  the  requirements  for  a trial  transit  route  as  described 
with  Figures  2-39  and  2-40,  the  user  would  be  presented  with  two  displays. 
Figure  2-41  - Transit  Area  Route  Chart,  which  shows  the  trial  transit  route 
superimposed  on  the  area  map,  is  presented  on  the  left  screen  and  Figure  2-42  • 
Route  Parameters  is  presented  on  the  right  screen.  Figure  2-42  shows  the 
Latitude,  Longitude  and  the  layover  time  (LO)  at  each  point.  The  Speed 
of  Advance  (SQA),  Distance,  Course  and  Time  between  successive  points  is 
also  shown.  In  the  sub-total  row  the  SQA  is  calculated  by  dividing  the 
total  distance  travelled  by  the  time  to  travel  each  leg  not  including 
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FIGURE  2-42  ROUTE  PARAMETERS 

layover  time.  In  the  total  row  SQA  and  total  time  includes  the  layover  time. 

If  the  user  desires  to  try  other  transit  routes  he  would  enter  N and  Figures 
2-39  and  2— 4o  would  again  appear  allowing  the  user  to  enter  different  parameters 
and  run  the  case  again,  it  would  be  more  convenient  for  the  user  if  a light 
pen  method  of  transit  route  entry  was  mechanized.  He  could  then  trace  trial 
routes  on  the  map  display,  avoiding  obstacles  or  high  threat  areas  (which  could 
also  be  displayed  on  the  map  background). 
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FIGURE  2-43  AVERAGE  THREAT  IEVEL 


The  user  may  also  elect  to  examine  the  average  hostile  threat  level  to 
vhich  the  Task  Force  will  be  exposed  for  a selected  transit  route.  To  do 
this  he  would  call  up  Figure  2-30  and  select  06  - AVERAGE  THREAT  I£VEL. 
Figure  2-44  - Average  Threat  Level  Prompt  would  appear  on  the  right  screen 
requesting  information  similar  to  that  described  in  the  discussion  of 
Figure  2-42.  A map  display  similar  to  Figure  2-4l  would  be  presented  on  the 
left  screen  on  which  the  user  could  trace,  with  a light  pen,  the  trial 
route  if  desired.  If  no  information  is  entered  for  the  transit  route 
coordinates,  the  route  last  defined  under  the  ROUTE  DEFINITION  category 
would  be  used.  Once  the  tentative  transit  route  is  defined.  Figure  2-43  - 
Average  Threat  Imvel  is  displayed  on  the  left  screen.  This  figure  shows 
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FIGURE  2-44  AVERAGE  THREAT  LEVEL  PROMPT 

the  grid  squares  with  their  respective  dot  densities  that  the  Task  Force  passes 
through  along  the  selected  transit  route.  Associated  with  each  grid  square 
is  a Figure  of  Merit  (FCM)  similar  to  Figure  2-37  which  represents  the  vulner- 
ability of  the  Task  Force  while  passing  through  that  square.  The  FGM's  are 
averaged  to  produce  a number  for  the  average  threat  level  the  Task  Force  will 
experience  for  the  selected  route.  This  number  is  displayed  in  Figure  2-44. 

The  user  may  select  alternate  transit  routes  for  evaluation  and,  as  noted  in 
Figure  2-44,  the  FGM's  from  both  the  previously  and  presently  selected  transit 
routes  are  displayed. 

I 
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FIGURE  2-45  RELATIVE  MOTION  DIAGRAM 


The  user  has  one  more  category  he  may  examine  to  aid  him  in  selecting  the 
best  transit  route,  and  that  is  the  use  of  a maneuvering  board  type  display 
to  analyze  relative  movement  situations,  typically,  between  the  Task  Force 
and  a threat.  A need  to  exercise  this  capability  may  arise,  far  example, 
if  information  is  received  that  an  Orange  Whiskey  Class  Submarine  has  been 
detected  along  the  planned  transit  route.  The  user  may  wish  to  avoid  the 
threat  by  altering  his  course  to  pass  the  submarine  no  closer  than  120  n.mi. 
to  remain  outside  his  missile  range.  Assuming  the  submarine  chooses  the 
optimum  course  for  intercept  and  closes  at  17  knots  while  the  task  force 
speed  of  advance  is  30  knots,  the  required  Task  Force  course  to  perform  this 
"end  run"  can  be  determined  as  well  as  tine  and  distance  to  CPA  (closest  point 
of  approach)  and  location  of  CPA.  Although  problems  of  this  type  would  be 
solved  by  the  conqmter  and  numerical  answers  displayed  as  in  Figure  2-46  - 
Relative  Motion  Parameters,  it  is  felt  a maneuvering  board  type  of 
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R - CPA  RANGE  (NMI) 
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TIME  TO  CPA  2 6.1  HR 
DISTANCE  TO  CPA  7S2  NMI 
LAT  AT  CPA  XX  ,XXN 
LONG  AT  CPA  XXX  ,XXE 


FIGURE  2-46  RELATIVE  MOTION  PARAMETERS 

presentation  shown  in  Figure  2-45  - Relative  Motion  Diagram  would  be  useful. 

Navy  personnel  are  experienced  with  maneuvering  board  displays  and  such 
a display  would  enable  such  users  to  determine  if  the  computer  solutions  are 
reasonable  and  would  give  them  increased  confidence  in  the  solutions.  Ry 
analyzing  the  maneuvering  board  display  a user  can  visually  estimate  the  relative 
motion  parameters  that  would  occur  with  alternate  inputs  of  course,  speed  and 
distance  without  having  to  actually  input  data  and  run  a computer  solution*  To 
use  the  foregoing  capability  the  user  would  call  up  Figure  2-30  and  select  C7  - 
REIATIVE  MOVEMENT.  Using  Figure  2-46,  he  would  enter  the  necesaazy  parameters, 
and  the  results  would  appear  as  alphanumeric s on  Figure  2-46,  and  as  graphics  on 
Figure  2-4$.  Of  course  many  other  types  of  relative  motion  problems  than  those 
illustrated  aay  be  solved  with  this  approach. 
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The  user,  having  utilized  the  capabilities  that  nay  be  exercised  by 
calling  up  any  or  all  of  the  seven  transit  route  selection  categories,  is 
now  in  a position  to  make  a decision.  Some  of  the  data  display  decision  aids 
he  has  examined  point  him  toward  a specific  transit  route  or  routes  consider- 
ing that  one  category  only  (e.g. , vulnerability).  However,  an  attempt  is  not 
made  to  combine,  through  the  computerized  mechanization,  the  results  from  all 
the  categories.  The  relative  importance  of  each  category  in  formulating  the 
final  decision  must  be  left  to  the  user. 


2.8  Proposed 


Sequences 


riment  to  Test  the  Efficacy  and  Acceptance  of  Data  Displa 


Experience  gained  in  the  course  of  current  effort  indicates  that  the 
technique  of  exposing  a test  population  comprised  of  experienced  naval  officer 
subjects  to  a sequence  of  displays  in  the  context  of  a prepared  scenario  is  a 
very  effective  method  to  establish  the  degree  of  utility  of  selected  presenta- 
tion modes  and  infoimation  foztaata  by  comparison  to  other  candidate  displays, 
and  the  degree  of  enthusiasm  associated  with  their  acceptance  by  operation- 
ally oriented  personnel. 

Further  experimental  evaluation  to  investigate  these  aspects  appears  to 
be  indicated.  Such  an  evaluation  program  would  comprise  the  following  major 
tasks : 

• Adaptation  of  existing  scenario  material  to  more  specifically 
define  and  highlight  the  courses  of  action  available  in  each  of 
two  involved  decisions  (Ops  Area  Select  and  Transit  Route  Select). 

e Preparation  and  formatting  of  supportive  data  using  each  of  four 
techniques 

Tabular  presentation 
Isogram  charts 

Ifianerically  labelled  configuration 
Dot  grid 

e Comparative  simulation  testing  by  a population  consisting  of  opera- 
tionally experienced  subjects  to  provide  both  objective  and  sub- 
jective evaluations  based  on  application  in  two  related  decisions. 
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• Restructuring  as  required  to  facilitate  the  derivation  of  a 
logically  "correct”  solution  for  control  purposes. 

• Elicitation  and  analysis  of  subject  reaction  and  comments. 

• Generation  of  report  describing  experimental  results. 

Since  a significant  portion  of  the  results  to  be  obtained  from  these 
tests  would  consist  of  subjective  evaluation  of  the  data  display  sequence 
methodology,  the  degree  of  experience  and  familiarity  of  the  test  subjects 
with  the  field  of  comnand  and  control  in  naval  operations  is  of  prime  im- 
portance. It  is  considered  that  a test  technique  utilizing  a population 
composed  of  experienced  senior  naval  personnel  in  an  interactive  war- room 
simulator  system  as  was  done  with  the  Options  Selection  Matrix  experiments 
provides  the  most  effective  approach  to  actual  shipboard  system  evaluation 
that  is  practically  attainable. 
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2.9  Conclusions  and  Recommendations 

A series  of  interactive  graphic  and  alphanumeric  displays  that  could 
assist  a user  in  formulating  decisions  in  two  specific  tactical  situations 
have  been  presented.  Although  details  of  the  software  requirements  to  gen- 
erate these  displays  are  not  discussed,  these  requirements  have  been  consid- 
ered as  the  displays  were  conceived.  Many  of  the  displays  (i.e.,  dot  grid, 
missile  coverage,  satellite  coverage,  time-distance-course  plots,  etc.) 
are  presently  operating  in  similar  form  in  the  ITFCC  program.  Grunina n devel- 
oped the  algorithms  and  code  necessary  to  implement  these  types  of  displays 
in  a DEC  PDP- 11/1*5  mini-computer  presently  installed  aboard  the  JEK  under 
contract  to  NAVELEX  on  the  ITFCC  program. 

The  following  factors  were  given  major  consideration  during  the  concep- 
tion of  decision  methodology. 

• Allow  maximum  utilization  of  the  user's  judgement  and  skills  by 
implementing  an  interactive  system  which  permits  the  user  to 
accept,  override  or  alter  the  processed  data. 

• Present  information  in  common  formats  and  procedures  to  expedite 
user  interpretation. 
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• Permit  the  user  to  proceed  directly  to  a computer  generated 

solution  for  minor  decision  points  in  a panic  situation  when  speed 
is  crucial  or  pennit  the  user  to  examine  many  detailed  facets  of 
the  problem  step-by-step  as  his  work  load  allows. 

Although  the  display  formats  were  structured  around  solving  two  specific 
decision  problems,  they  have  a broad  spectrum  of  applications,  especially  where 
geographical  considerations  are  involved. 

This  report  has  presented  data  displays  as  they  would  appear  and  be  used 
by  operational  personnel.  The  next  step  would  be  to  develop  new  and  modify 
existing  software  necessary  to  generate  these  displays  and  provide  the  inter- 
active capability.  At  this  point  the  computer  sizing  requirements  and  the 
cost  of  implementing  a demonstration  model  could  be  determined.  The  conclud- 
ing step  would  be  to  implement  a demonstration  system  and  conduct  a valida- 
tion test  as  discussed  in  paragraph  2.8. 
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